Re–evaluation of the cost–effectiveness and effects of childhood rotavirus vaccination in Norway

Background Rotavirus vaccination was included into the Norwegian childhood immunisation programme in 2014. Before implementation, rotavirus vaccination was found to be cost–effective from a societal perspective, but not from a healthcare perspective. Since introduction, new data on the incidence and economic effects of rotavirus disease have become available. We assessed early epidemiological effects of the rotavirus vaccination programme and re–evaluated its cost–effectiveness in Norway for the years 2015–2019. Methods Using a dynamic transmission model, we compared the epidemiological effects of the ongoing two–dose vaccination programme with Rotarix®, and a hypothetical 3–dose programme with RotaTeq® with no vaccination. A baseline cost of € 54 per fully vaccinated child was used. Cost–effectiveness was computed from a healthcare and societal perspective, using a decision analytical model. Data on healthcare use and costs, productivity losses and health utilities were based on published and own estimates. Uncertainty was accounted for in one–way, multi–way, and probabilistic sensitivity analyses. Results During 2015–2019, 114,658 home care cases, 34,571 primary care cases, 7,381 severe cases, and 2 deaths associated with rotavirus disease were avoided due to vaccination. Under baseline assumptions vaccination was cost–effective from a healthcare perspective with a cost per QALY of € 47,447 for Rotarix® and € 52,709 for RotaTeq®. The break–even price was € 70 for Rotarix® and € 67 for RotaTeq®. Vaccination was cost–saving from the societal perspective, and also from a healthcare perspective for vaccine prices below € 25 and € 22 per vaccinated child for Rotarix® and RotaTeq®, respectively. Conclusion Ongoing childhood rotavirus vaccination in Norway has reduced the rotavirus disease burden substantially, and is cost–effective compared with no vaccination.


Results
During 2015-2019, 114,658 home care cases, 34,571 primary care cases, 7,381 severe cases, and 2 deaths associated with rotavirus disease were avoided due to vaccination. Under baseline assumptions vaccination was cost-effective from a healthcare perspective with a cost per QALY of 47,447 for Rotarix ® and 52,709 for RotaTeq ® . The break-even price was 70 for Rotarix ® and 67 for RotaTeq ® . Vaccination was cost-saving from the societal perspective, and also from a healthcare perspective for vaccine prices below 25

Introduction
Rotavirus gastroenteritis (RVGE) leads to a considerable health and economic burden in unvaccinated populations [1]. In developing countries, mortality from rotavirus is common, while in high-income countries rotavirus-associated deaths are rare. [2,3] The disease is largely preventable through vaccination [4], and in recent years, rotavirus vaccines have been introduced in vaccination programmes of 81 countries worldwide; eight of which are within the European Union. [5] Many studies clearly illustrate the health benefits of rotavirus vaccination, but whether these benefits outweigh the costs of vaccination is country-specific [6]. In Norway, the rotavirus vaccine was included in the national childhood immunisation programme in 2014. Two rotavirus vaccines are currently licenced for use in Norway: Rotarix 1 (GlaxoSmithKline Biologicals, Rixensart, Belgium) and RotaTeq 1 (Merck & Co., Inc., Whitehouse Station, N.J., U.S.A.). Rotarix 1 is a human G1 monovalent vaccine, requiring two separate doses per child, while RotaTeq 1 is a bovine-human reassortant pentavalent vaccine requiring three separate doses per child. As a part of the Norwegian childhood vaccination programme, Rotarix 1 is offered free of charge to all children born on or later than 1 September 2014.
Our study is the first to describe the early effects of the ongoing rotavirus vaccination programme, and is the second study to evaluate the cost-effectiveness of rotavirus vaccination in the Norwegian setting. The previous health economic evaluation of universal childhood rotavirus vaccination concluded that vaccination would be cost-effective from a societal perspective, but not from a healthcare perspective [7], but this study was limited by the availability of data. The incidence of hospitalisation due to RVGE was estimated at three cases per 1000 children below 5 years of age [8], while the incidence of primary care consultations and homecare disease episodes was derived from the hospitalisation estimates. Since 2009, new and improved studies have assessed the health [9] and economic [10] burden of rotavirus in the primary and secondary healthcare sectors in Norway. These studies reported a higher incidence of rotavirus-associated primary care consultations (30.6 per 1000 children) and hospitalisations (6.3 per 1000 children) [9]. Rotavirus-associated mortality was estimated at 0.17 deaths per 100,000 children under 5 years of age [9]. These recent data suggest that the disease burden of RVGE was underestimated in the previous economic evaluation; therefore a re-evaluation is warranted. Since the vaccination programme has already been initiated, we were able to use new data on the burden of disease after vaccine introduction [9,11]. We also applied new data concerning workdays lost among caregivers of hospitalized rotavirus cases to better estimate productivity losses [12].
In this study we combine data from the pre-and post-vaccination periods to estimate the initial epidemiological effects of the ongoing rotavirus vaccination programme in Norway, and re-evaluate the cost-effectiveness of the vaccination programme compared with no vaccination from the healthcare and societal perspectives.

Study setting
Norway is located in Scandinavia and has a population of 5.2 million inhabitants [13]. The Norwegian healthcare system is mainly financed by the public sector and provides universal coverage to all residents. The childhood vaccination programme is included in this scheme, and all children are eligible to receive programme vaccines free of charge. Some private healthcare services are available, but the majority of hospitals are public. The healthcare system includes a primary and a secondary sector. Primary care has a gatekeeping role to secondary care. The primary care sector is organized at the municipal level, whereas secondary care is administered at the regional level. Norwegian parents have a right to 10 workdays of leave each, with full income compensation, to care for their ill children under the age of 12 years. Parents of multiple children and parents with single custody are entitled to additional days of leave. [14] Mathematical model An extended version of a previously published [15][16][17] dynamic rotavirus model was adapted to the Norwegian setting. The model has an M-SEIRS (Maternal immunity, Susceptible, Exposed, Infected, Recovered, Susceptible) structure and includes separate modelling of the first, second, third and later rotavirus infections. We stratified the population into 74 age groups in the following way: <5 years (monthly age groups), 5-69 years (5-yearly age groups), and !70 years. Social mixing in the model was based on physical contact data from Finland collected as a part of the POLYMOD study on social contact patterns [18]. We fitted our model using weekly hospital sentinel data from March 2006 to February 2008 and from January 2014 to February 2015; 417 children aged <5 years of age with laboratory confirmed rotavirus infection were included. Vaccination began in mid-October 2014 and we assumed that vaccine-related changes in the rotavirus epidemiology before February 2015 were negligible. Due to the limited and time-interrupted data, we collapsed the sentinel data into a single year based on the weekly incidence of rotavirus prior to model fitting (S1 Fig). In the model fitting procedure we applied a burn-in period of 40 years to let the model reach a steady state using historic and projected data on births, mortality, and net immigration rates from 1970-2019 [13]. We estimated the average transmission rate by maximizing the likelihood under the assumption that the count of laboratory confirmed hospitalization events in each age group <5 years were multinomial distributed, as detailed by de Blasio et al. [16]. The model fit was repeated for all possible combinations of: 1) two different values for the infectivity of later infections; 2) reduced mixing in children <12 months, corresponding to the duration of the maternity leave in Norway; 3) reduced hospitalization rate for children >2.5 years with severe RVGE compared to younger children. In total, eight different models were tested. The final model included one parameter representing the relative reduced mixing among children under 12 months of age, and one parameter representing a reduced reporting rate in children above 2.5 years of age, which improved the maximum likelihood estimation model fit (S1 Table). Seasonality was incorporated in the model by including a sinusoidal forcing term to the transmission rate with a period of one year. Because the weekly sentinel observations were few, we fitted the amplitude and phase of the seasonal forcing separately and after the primary model selection.
Infections were classified as mild, severe or asymptomatic in accordance with Velázquez et al. [19], and the duration of immunity following natural infection was assumed to be one year [15,[20][21][22]. The susceptibility and infectivity was assumed to be reduced during the second and later infections relative to the first infection [19], ( Table 1). The majority of rotavirus Uniform random (0.225-0.375) [26] Proportion severe RVGE (among children with no previous infection) b Uniform random (0.01-0.04) [26,27] Mean duration of vaccine-induced immunity (years) 2 [23,25,27,32] Coverage (max after introduction) 0.947 first dose (baseline); 0.877 second dose [11] Healthcare and deaths in children under five years of age cases classified as severe were all assumed to require primary healthcare consultations [12]; the mean proportion was assumed to be 0.975 (range 0.95-1.0). Hospitalisations and deaths among children with severe infections were calibrated to recent Norwegian national estimates published by Bruun et al. [9]. Mild cases were assumed to be cared for at home without medical attention.
We simulated the impact of the national vaccination programme starting on 14 October 2014 until the end of 2019. In total, 10,000 simulations were run using the fitted model with random values for primary care consultations, hospital inpatient and outpatient contacts, and deaths; stochastic simulation was implemented to estimate healthcare encounters. Results were compared with the models' projections without vaccination for a five-year period from 1 January 2015 until the end of 2019.
Vaccination was modelled as a two-dose regimen with Rotarix 1 in accordance with evidence from clinical trials suggesting that vaccine-induced protection lasts for more than one year [23,24], we assumed that the duration of vaccine induced protection was on average 2 years. In the base-case we assumed that the vaccine provided protection to all children after they received the first dose and that protection was achieved was at two months of life [25]. The proportion of severe symptomatic and symptomatic infections in children (with no prior infection) was assumed random with mean values of 0.025 and 0.300, respectively. These numbers were estimated to target vaccine efficacy values consistent with current estimates from high-income countries; the mean effectiveness against severe and symptomatic infection were 0.93 and 0.60, respectively [26,27]. The vaccine was assumed to have an equal effect on susceptibility and infectivity as that of a natural infection [21] ( Table 1).The effect of the second dose was not explicitly modelled. A logistic growth curve model was used to estimate rotavirus vaccination coverage during the first five years after the programme start. The model was simulated using real-world vaccination coverage rates observed in Norway during September 2014-March 2016 [11]. Coverage estimates with 95% confidence intervals were predicted by the model for the remaining study period of April 2016-December 2019.

Cost-effectiveness study
The economic evaluation was based on the number of rotavirus inpatient and outpatient hospital cases, primary care consultations, and deaths estimated by the dynamic model. We evaluated the costs and effects of vaccination and no vaccination for the period 2015-2019. Because the phasing-in period in 2015 is relatively ineffective, and unlikely to be representative of coming years, we conducted a separate analysis for the period 2016-2019. For each vaccination strategy, we computed healthcare costs, productivity losses due to work absences, and healthrelated quality of life detriments. In the vaccination strategy, we added the costs of the vaccination programme, including the vaccine itself, and also implementation, administration and storage costs. To compute the costs of primary care consultations more accurately, we differentiated between primary care consultation at emergency outpatient clinics (EOCs) and general practitioner (GP) offices based on proportions reported by Shin et al. [10].
Health economic outputs were expressed as cost per quality adjusted life year (QALY), cost per case avoided, and cost per severe case avoided. Cost-effectiveness was expressed in terms of an incremental cost-effectiveness ratio (ICER) from healthcare and societal perspectives, and was assessed separately for Rotarix 1 and RotaTeq 1 . We have used the threshold value of 73,444 (NOK 657,540) as advised in the Norwegian guidelines for health economic evaluations [33]. This value was adjusted to 2015 price levels by accounting for changes in households' real disposable income. All costs were converted from Norwegian Kroner to 2015 , using the average exchange rate during 2015 [1 = 8.953 NOK] [34]. A yearly discount rate of 4% was applied to costs and health effects, as recommended by the Norwegian guidelines [33].
Healthcare costs. To estimate the cost of telephone calls and consultations with GPs, EOCs, and hospital inpatient and outpatient contacts we used the methods and data described by Shin et al. [10], but updated the estimates with Norwegian 2015 diagnosis related group (DRG) prices for hospital-based costs [36], and with GP reimbursement rates adjusted to 2015 price levels using the consumer price index [13].
Indirect costs. For children <1 year of age we assumed that no workdays were lost by parents because the parental leave in Norway lasts from 49 to 59 weeks. Ill children aged one year or older were all assumed to require parental care. Based on a recent survey of caregivers of Norwegian children <5 years of age hospitalised with RVGE, the number of workdays lost by caregivers was 3.25 days per child [12]. Based on expert opinion and assumptions in similar studies, the duration of caregiver work loss for non-hospitalized children requiring medical attention was assumed to be 1.5 days, and caregivers of homecare cases were assumed to lose one day [37][38][39]. A human capital approach was used to value workdays lost, with labour costs computed based on the average full-time equivalent wage of caregivers, added the costs not returned to the worker [13].
Vaccination costs. Vaccines included in the national immunisation programme in Norway are procured through the national tender, consequently, the price of programme vaccines is typically somewhat lower than pharmacies' maximum retail price [40]. Accordingly, the mean baseline vaccine cost per vaccinated child was set to 54, both for Rotarix 1 and Rota-Teq 1 . The time spent administering the vaccine was based on reports from immunisation nurses (unpublished data), and was higher for RotaTeq 1 compared with Rotarix 1 due to an additional dose. Administration time was assumed to decline with the number of doses given, due to parental familiarity with the vaccine. The value of time spent on vaccine administration was estimated based on the average labour cost of public health service nurses, since specific salary data for immunisation nurses were unavailable [13]. The cost of storing the national rotavirus vaccine stockpile was estimated at 6,953 per year, while the costs of storage at administration venues were not included. We estimated a yearly cost of 335,083 to account for operational and implementation costs of the vaccination programme. This was assumed to be equal to the budgeted expenses associated with rotavirus vaccination in 2016. Vaccination costs are reported as the cost per vaccinated child, implying two-doses for Rotarix 1 and three doses for RotaTeq 1 .
Quality of life measures. Utility estimates were based on the UK estimates reported by Marlow et al. [35] and these were based on the Health Utilities Index 2 [41] for children, and EQ-5D-5L [42] for parents of inpatients and outpatients [35]. For non-hospitalised cases, we assumed a QALY loss ranging from 0 to the lower confidence interval of reported utility losses for outpatients. Detriments due to mortality were computed based on Norwegian data concerning age-specific losses in expected remaining life years, with a yearly discount rate of 4%.
Sensitivity analyses. To account for parameter uncertainty, we conducted Monte Carlo simulations with 10 000 iterations on each parameter using the distributions reported in Table 2. A one-way sensitivity analysis was performed on all key parameters, varying each by ± 20%. Threshold analyses were done for the baseline scenarios to evaluate cost-effectiveness of vaccination at a variety of vaccine prices for Rotarix 1 and RotaTeq 1 . These analyses were only done from the healthcare perspective.
To test the robustness of our findings we conducted a range of scenario analyses to explore the effects of unknown and uncertain parameter values in the model. To simplify comparisons between the different scenarios we report cost-effectiveness in terms of net monetary benefit (NMB), in which costs and effects are combined into a single monetary value using a predefined threshold value ( 73,444). NMB = (effects x threshold value)-costs. In addition to the baseline scenarios, we evaluated the effect of I) reducing productivity losses due to caregiving from 100% to 50%; II) including parental QALY losses in addition to QALY losses for ill children; III) removing discounting on costs and effects; IV) including the impact of mild adverse events after vaccination by assuming a quality of life loss equal to half of that of mild cases for 1 in 50 vaccinated children; V) assuming vaccine protection to depend on two vaccine doses using the vaccine coverage of the second dose and effective protection in children at four month of age, instead of two months of age in the base case; VI) excluding utility effects from avoided deaths as an attempt to balance out the effect of more severe adverse events such as intussusception. Finally, we assessed cost-effectiveness under best-and worstcase scenarios by conducting multi-way analyses of the best and worst combinations of scenarios I-VI.

Epidemiological effects
During the first five years following introduction (2015-2019), the number of hospitalised rotavirus cases and deaths was reduced by 73%, the number of primary care cases was decreased by 70%, and the number of home cases was reduced by 64% (Table 3, Fig 1). The reduction was greatest for cases between one and two years of age, with 45% of severe cases and 32% of mild cases occurring in this age group. (S3 Fig). In 2015, the total number of avoided symptomatic cases (6,302) was lower than for years between 2016 and 2019 wherein the total number of avoided symptomatic cases varied from 24,302-37,327 per year. Reductions in healthcare resource use corresponded with the number of mild and severe cases avoided. The relative reductions in healthcare use was greatest for hospital inpatient and outpatient contacts, while the absolute reduction was greatest for primary care consultations ( Table 4).
The model also predicted changes in the rotavirus seasonal pattern post-introduction. The changes included the appearance of biannual peaks in rotavirus incidence, instead of annual peaks, which were present during the pre-vaccine era (Fig 2 and S3 Fig).

Cost-effectiveness
In the first five years after introduction, childhood rotavirus vaccination was predicted to be cost-effective from a healthcare perspective and cost-saving from a societal perspective (Table 5). Under baseline vaccines prices, Rotarix 1 was less costly and thus more cost-effective than RotaTeq 1 . For all years between 2016 and 2019 vaccination was cost-saving from the societal perspective, and cost-effective from the healthcare perspective, but in 2015 vaccination was not cost-effective from the societal or healthcare perspective (Fig 3).  During the first five years after vaccine implementation, healthcare costs were reduced by 72%. The absolute cost reduction was greatest for inpatient costs, followed by GP consultations, outpatient hospital contacts and EOC consultations ( Table 6). Productivity losses were decreased by 61%, and the largest absolute reductions in productivity losses were a result of fewer homecare cases (Table 6). QALY losses went down by 70%, with the greatest absolute reduction in morbidity (Table 6). Under baseline assumptions, the mean yearly vaccination cost per year for the period 2015-2019, was 185,624 higher for RotaTeq 1 compared with Rotarix 1 (Table 6).

Sensitivity analyses
One-way sensitivity analysis. Varying parameters associated with mild rotavirus cases, namely, the health-related quality of life and workdays lost associated with homecare cases Cost-effectiveness of childhood rotavirus vaccination in Norway had the greatest impact on cost-effectiveness in the one-way sensitivity analyses (Fig 4). In addition, cost-effectiveness of the programme was sensitive to variations in the number of children vaccinated, vaccine price, and inpatient hospital costs (Fig 4).
Scenario analyses. In all scenario analyses, vaccination remained a cost-effective strategy (Fig 5). From the healthcare perspective, the best-case scenario resulted in a mean NMB of 3,728 thousand, while the worst-case scenario resulted in a NMB of 297 thousand (Fig 5A). In the societal perspective, the best-case scenario resulted in a NMB equal to 12,148 thousand, and in the worst-case scenario, the NMB was 4,169 thousand ( Fig 5B).
Threshold analyses on vaccine prices. Under baseline assumptions, vaccination was cost-saving from the healthcare perspective for vaccine prices of 25 or lower for Rotarix 1 and 22 for RotaTeq 1 . The break-even price per fully vaccinated child for the period 2015-2019 was 70 for Rotarix 1 and 67 for RotaTeq 1 , on average for the period 2015-2019 (Fig 6A and  6B). For the post-introduction period of 2016-2019, the break-even price was 7.5 higher for both vaccines (Fig 6B and 6D).
Acceptability curves. During 2015-2019, the likelihood that a rotavirus vaccination programme was cost-effective under baseline assumptions was 71% for Rotarix 1 and 68% for RotaTeq 1 from the healthcare perspective (Fig 7A). In the period of 2016-2019 the probability that a vaccination programme was cost-effective from the healthcare perspective was 88% for Rotarix 1 and 84% for RotaTeq 1 (Fig 7B).
From 2015-2019, a vaccination programme with Rotarix 1 was 86% likely to be cost-effective and a programme with RotaTeq 1 was 85% likely to be cost-effective from the societal perspective. During the post-introduction period, the likelihood of a vaccination programme with Rotarix 1 and RotaTeq 1 being cost-effective from the societal perspective was 100%. Cost-effectiveness of childhood rotavirus vaccination in Norway

Discussion
The burden of rotavirus vaccination in Norway has been reduced considerably following the implementation of the rotavirus vaccine, and according to our predictions, the burden will remain low during the coming years. Compared with a no vaccination strategy, rotavirus vaccination was predicted to result in a 72% reduction in rotavirus-related costs during 2015-2019. In addition, productivity losses associated with parental caregiving during illness episodes among children were expected to be 61% lower, and QALY detriments due to rotavirus were estimated to decrease by 70%. We found that the ongoing vaccination programme was cost-effective from both a healthcare and a societal perspective. In the baseline scenario, vaccination was cost-saving from a societal perspective for a wide range of vaccine prices, whereas from a healthcare perspective, vaccination was cost-saving for vaccine prices equal to or lower that 25 and 22 per fully vaccinated child for Rotarix 1 and RotaTeq 1 , respectively. Vaccination was also found to be cost-effective from the healthcare and societal perspectives under all the sensitivity analyses considered in our study.
Comparing findings between countries is problematic due to country-specific variations in costs, availability of country-specific data, methodology, and national guidelines for health economic evaluations. Cost-effectiveness of rotavirus vaccination programmes varies between developed countries [6], but also within individual countries [43]. However, our cost per QALY values and break-even vaccine prices are similar to those reported by other countries https://doi.org/10.1371/journal.pone.0183306.g007 [6,43]. In Norway, the threshold willingness-to-pay for a QALY is relatively high ( 73 444) in comparison with other countries, which results in vaccination being considered cost-effective in our study. Previous studies have used a threshold value of 30 000 to compare cost-effectiveness across nations; if we apply this threshold value in our study, rotavirus vaccination would be unlikely to be cost-effective (Fig 7), and the break-even price would be lower (Fig 6).
Health economic evaluations of vaccination programmes should ideally be performed prior to vaccine implementation. However, a clear advantage of conducting a post-introductory cost-effectiveness evaluation is access to good-quality data from both before and immediately after implementation. For instance, we based our model estimates of vaccine coverage on realworld observed data as the programme was rolling out. To fit the model, we used data from hospital-based surveillance of acute gastroenteritis cases, which captured changes in the rotavirus incidence in real time and thus allowed timely monitoring of the effects of the rotavirus vaccination. Surveillance data on work absenteeism and healthcare resource use prior to hospital admission were also used to calculate productivity losses and use of healthcare resources in the economic model [12].
Our acceptability curves and a wide range of sensitivity analyses illustrated that rotavirus vaccination in Norway is likely to be cost-effective. Our study should however be interpreted with the following limitations in mind. We conservatively assumed the duration of vaccineinduced protection to be on average two years. Direct vaccine-induced protection may be longer, as suggested by other studies [23,24]. The duration of natural immunity was assumed to be on average, one year, but this value is in line with the literature [15,[20][21][22]. The potential economic benefits resulting from herd protection among persons above five years of age [44] were not included in this model, and we may therefore have underestimated the benefits of vaccination. We also lacked Norwegian data on social mixing in the population. Finnish data were used as a substitute, but these data may not be fully representative of the Norwegian population. Our incidence and cost data on hospitalised rotavirus cases were of high quality, but we lacked information about nosocomial rotavirus cases. We could not identify any studies on the burden of nosocomial RVGE cases for the Norwegian setting, and available international studies have been found to report highly variable results depending on the specific methodologies applied [45]. For some model parameters, particularly parameters relating to milder rotavirus cases, we made assumptions based on published data from other settings, and although extensive sensitivity analyses were performed to evaluate the impact of variation in these parameters, this may have impacted the accuracy of our findings. Due to a lack of data we did not include costs of consumables, travel to and from healthcare facilities, or vaccine wastage costs (due to expiry, cold-chain failures, or children regurgitating the vaccine).
Productivity losses associated with parental caregiving of RVGE cases are substantial. In the baseline, we assume zero productivity during the caregiving period. We did not have information about the proportion of parents working part time, or teleworking. To compensate for this we assessed the impact of assuming 50% lower productivity loss during the period in which caregivers were absent from work due to child's illness. In this scenario, rotavirus vaccination remained cost-saving from the societal perspective, but savings from vaccination were lower. The effect of different concurrent infections among children within a household [46] were not accounted for, therefore the average number of workdays lost may have been slightly overestimated. On the other hand, we may have underestimated the productivity losses because we did not include work loss following the death of a child. While productivity losses following child deaths have been shown to be considerable [47], we were unable to find quantified estimates of workdays lost following the death of a child.
In our study, some of the main uncertainties were a result of limited data to estimate certain parameters. Future studies should take particular care to assess the costs and health consequences among mild rotavirus cases, as these cases account for a considerable proportion of the total rotavirus burden. In addition, once vaccinated cohorts' age, data should be collected to assess the long-terms effects of rotavirus vaccination; and new health economic evaluations should be conducted.

Conclusion
Universal childhood rotavirus vaccination in Norway is expected to reduce the rotavirus disease burden considerably, and vaccination can be considered a cost-effective strategy from the societal and healthcare perspectives under the assumed threshold value.