Inclusion of Safety-Related Issues in Economic Evaluations for Seasonal Influenza Vaccines: A Systematic Review

(1) Background: Vaccines for seasonal influenza are a good preventive and cost-effective strategy. However, it is unknown if and how these economic evaluations include the adverse events following immunization (AEFI), and what the impact of such inclusion is on the health economic outcomes. (2) Methods: We searched the literature, up to January 2020, to identify economic evaluations of seasonal influenza vaccines that considered AEFIs. The review protocol was published in PROSPERO (CDR42017058523). (3) Results: A total of 52 economic evaluations considered AEFI-related parameters in their analyses, reflecting 16% of the economic evaluations on seasonal influenza vaccines in the initial study selection. Most studies used the societal perspective (64%) and evaluated vaccination of children (37%). Where considered, studies included direct medical costs of AEFIs (90%), indirect costs (27%), and disutilities/quality-adjusted life years loss due to AEFIs (37%). The majority of these studies accounted for the effects of the costs of AEFI on cost-effectiveness for Guillain–Barré syndrome. In those papers allowing cost share estimation, direct medical cost of AFEIs was less than 2% of total direct costs. (4) Conclusions: Although the overall impact of AEFIs on the cost-effectiveness outcomes was found to be low, we urge their inclusion in economic evaluations of seasonal influenza vaccines to reflect comprehensive reports for the decision makers and end-users of the vaccination strategies.


Introduction
Seasonal influenza spreads very easily among people from all age groups almost every year [1]. Although influenza is mostly self-limiting, serious complications can arise in vulnerable patient groups. Populations with higher risk of complications include pregnant women, patients with chronic medical conditions, children aged 6-60 months, and the elderly [1,2]. Vaccination is considered as the most effective way to prevent disease and/or severe outcomes from the illness [1]. Since 1945, influenza vaccines have been marketed and used as an efficient and cost-effective preventive tool [3].
There are several types of seasonal influenza vaccines available worldwide. Most common are inactivated influenza vaccines (IIVs), available as trivalent influenza vaccines (TIVs) or quadrivalent influenza vaccines (QIVs), and live attenuated influenza vaccines (LAIVs), which are nowadays only available as quadrivalent vaccines (Q-LAIVs). While

Information Sources and Search Strategy
We searched for full-text published studies in peer-reviewed journals. Our focus was on full economic evaluations of seasonal influenza vaccines. The search strategy was performed in 3 steps. First, we performed an initial limited search (using the following keywords: influenza, vaccine, seasonal, and economics) of MEDLINE in PubMed, followed by analysis of the text words contained in the title, the abstract, and the index terms used to describe the articles. This was used to create a detailed search strategy given in Table 1, including 4 search lines, each composed of Medical Subject Headings (MeSH) terms, as well as other, previously identified index terms.
In the second step, we ran the 4 search lines (as presented in Table 1) in MEDLINE, EMBASE, and the Cochrane Library to search for articles of interest. In the third step, we searched the reference lists of all identified reports and articles for additional studies. We also checked gray literature (Google search, governmental and research reports), and searched additional databases indexing economic evaluations, such as National Institute for Health Research Economic Evaluation Database (NHS EED) and Health Economic Evaluations Database and Cost Effectiveness Analysis registry (HEED CEA), to ensure a comprehensive coverage. Within these databases, applying the detailed search strategy was not possible, and therefore we performed searches using only the key words "influenza vaccines" and assessed those results for adherence to our selection criteria in order to add eligible articles for a final review. Studies published until January 2020 were considered for inclusion in this review. No language limitation was applied.

Study Selection and Data Collection Process
Two researchers, T.F. and P.T.d.B., independently assessed the search results-1152 articles. Disagreements were discussed and consensus was reached with the help of M.J.P.

Study Selection
The search results were examined in 2 rounds, using a unified study selection form (Appendix A, Table A1) that included 5 selection criteria questions. In the first round of title/abstract screening, 4 out of the 5 selection questions were considered to identify the eligible economic evaluations. In the second round of full text screening, we addressed the fifth question as well, which allowed us to identify the economic evaluations that discuss AEFIs. Agreement/reliability was calculated using kappa statistics [19].

Data Extraction and Data Items
Data extraction was conducted using a pre-designed data extraction form (Supplementary Materials, Table S1) by the review team. This form includes information about study identification, type of economic evaluation, study perspective, type of vaccine used and its comparator, population characteristic (e.g., health condition, gender and age), location/geographical information, settings, time period (year/time frame), cost data (total and AEFI-related costs), currency used, pricing period/date, AEFIs considered, other AEFIrelated data (e.g., health related quality of life due to AEFI, frequency of AEFI-occurrence and duration of AEFI), data sources, funding, authors' conclusions, and other relevant data identified as significant for inclusion during the article screening.

Outcomes
The primary outcomes considered AEFI-related costs and health effects, for example, the impact on utilities or health-related quality of life. Cost outcomes were expressed in 2020 price year levels using the Campbell and Cochrane Economics Methods Group and the Evidence for Policy and Practice Information and Coordination Centre web-based tool [20] to enhance comparability between studies. The percentage shares of AEFI-related costs in the total costs presented in the economic evaluations were studied as a secondary outcome.
According to the WHO, the AEFI is any untoward medical occurrence that follows immunization and that does not necessarily have a causal relationship with the usage of the vaccine [21]. The term AEFI was chosen to be used for this study in order to avoid misinterpretations due to the diverse presentation of the safety-related parameters within the studies, as investigating the casualty was not within the aim of this review.

Risk of Bias/Quality Assessment
After the study selection, 2 researchers, T.F. and P.T.d.B., assessed the quality of the study and the risk of bias. For the methodological quality, we used the Consensus Health Economic Criteria (CHEC), an extended list for conducting systematic reviews on the basis of economic evaluation studies [22,23]. This 20-question structure from the CHECextended list was filled with agreements and disagreements, as the list contains only yes/no questions. In case of insufficient information, or lack of clarity, we answered negatively, which does not indicate a complete exclusion of the concerned issue. When we had agreed that the issue in question was sufficiently described, we selected "yes". Disagreements regarding this assessment were resolved by involving a third researcher (M.J.P.). For better presentation of the outcomes, we inputted the questions from this checklist in the review production tool RevMan 5.3 (Review Manager 5.3, The Cochrane Collaboration, London, United Kingdom) [24]. This tool allows for selecting low/high/unclear risk of bias. Considering our checklist, we used "low risk" for our positive answers, "high risk" of bias for our negative answers, and "unclear risk" for items not applicable for the evaluated study.

Study Selection
We identified 1827 articles, from which 771 were in MEDLINE, 989 in EMBASE, and 67 in Cochrane Library. The duplicates were removed using RefWorks and further manual scanning of the abstracts, and this resulted in 675 duplicates from the three databases. The resulting number of unique articles for title/abstract selection was 1152. After the first round of title/abstract scanning (accounting for the first four questions of our study selection form given in Appendix A, Table A1), we identified 197 potentially suitable papers, which we further explored within their full text (with the fifth question of the study selection form given in Appendix A, Table A1) for AEFI inclusion. The kappa statistic calculations resulted in a coefficient of 0.98, indicating good reliability according to Landis and Koch [25] and very good according to Altman's [26] interpretation (calculations are given in Appendix A, Table A2). This search, performed in the three above-mentioned databases, resulted in 31 articles eligible for our review (16% of the 197 full-text scanned papers). After searching the additional databases indexing economic evaluations as mentioned in the methods section, we added 21 articles that were eligible to be included in the review and adhering to the selection criteria. These studies were not presented through the steps of selection, as they were not identified using the detailed search strategy (given in Table 1), and therefore the 21 eligible studies from the additional search were directly added to the 31 eligible studies from the main database search. Finally, this review includes 52 (31 from the main search plus 21 from the additional search) full economic studies discussing safety/AEFI-related outcomes. The flow diagram of the study selection is presented in Figure 1.  CBA-cost-benefit analysis; EE-economic evaluation; AEFI-adverse event following immunization. This diagram shows the study selection process in steps, starting from database search, then removing duplicates, selecting by abstract screening, and full-text screening. The listed criteria for eliminating the full-text articles concern the following questions: (1) Is the article a full economic evaluation study (designs to be considered: CMA, CEA, CBA, or CUA)? (2) Is the intervention a vaccination? (3) Is the vaccine used for seasonal influenza? (4) Are the outcome measures economic parameters? (5) Does this EE discuses AEFI? Bold: 31+21 = 52, the total number of reviewed studies.

Study Characteristics
The general study characteristics and the different ways our reviewed studies reflected these characteristics are summarized in Table 2, including the percentage of studies reflecting those specific characteristics in the total reviewed articles. Further details regarding the extracted data, as well as the AEFI definitions, AEFI sources, ways the AEFI were ascertained, and in what time period, can be found in Supplementary Materials, Table S1.

Data Sources and Funding
Data about the costs of AEFIs were derived from publications and public databases.
For example, few studies [59,73,77,78] used cost data based on the database (The Medstat Group) reporting payments for health insurance companies in the USA [42,43]. The IBM Micromedex RED BOOK [80] was also often used as a cost reference source [30,32,[36][37][38][39][40]42,49,51,55,78]. Health impact data were mostly taken from population-based studies and surveys (utilities/quality-adjusted life years (QALYs)) [53,81], and one study [39] used an estimate for the QALYs. Clinical trials were mostly the source for probabilities/frequencies. The majority of the funded studies were financed by national or international health institutions, while only 15% of those studies received financial support from the pharmaceutical industry.
Another study [53] used this equitation to present unit costs: • direct AEFI-related unit cost = payment for one outpatient treatment × rate of AEFI due to vaccination × half of these need outpatient treatment; • indirect AEFI-related unit cost = rate of AEFI × 1/2 accompanying person × payment for 1/2 outpatient treatment × time lost × productivity loss (cost per hour); Therefore, straightforward comparisons of AEFI costs were not possible, and these results are given per individual study (see Appendix A, Figure A1 and Supplementary Materials, Table S1).

AEFIs' Share of the Total Costs Discussed in the Economic Evaluation
Not all papers allowed for an estimation of the share of AEFIs in the total costs discussed in the economic evaluation. We identified only four articles [52,56,69,73] where it was possible to give a clear reflection of AEFIs' direct medical costs in the total direct medical costs. In three of these papers, direct costs of AEFIs reflected less than 1%, and one less than 2% of total direct costs. They concern the vaccine-related AEFI management [73] and treatment of AEFIs [52,56,69], representing 0.05%, 0.34%, 0.79%, and 1.8% of the total direct costs, respectively.

Quality Assessment Using CHEC Extended List
The graphical presentation of the judgments about the quality assessment items from the CHEC extended list is given in Appendix A, Figure A2. This assessment showed that the generalizability is worst reported. Moreover, the time horizon reporting reflected poor results. As the majority of the studies reported time horizon of one year or one season, we found that period not long enough to capture all the AEFI outcomes if the study was not in trial settings, or especially if it considered the societal perspective. On the other hand, all Vaccines 2021, 9, 111 16 of 28 articles had well-defined research questions and appropriate study design, clearly described the study population, and properly presented the conclusions following the reported data. Overall, we found the reporting quality of the papers to be at a satisfactory level. Risk of bias for each individual study accounting the 20 questions form the CHEC extended list for quality assessment of economic evaluations is given in Appendix A, Figure A3.

Main Findings
We conducted this study to investigate if and how the economic evaluations on seasonal influenza vaccines include the AEFIs, and further explored how are they used in the analyses considering this issue. The results showed that 16% of the studies we initially considered for full-text screening included effects of AEFI within the economic evaluation. However, among the studies that accounted for AEFI, no consistent and comprehensive reporting of AEFI was noted. In this matter, costs and utilities of rare and expensive AEFIs, such as GBS, seemed to be more interesting for inclusion in the economic evaluation on influenza vaccines than the more frequently occurring but mild AEFIs. While the reporting of AEFIs costs was preferred in a matter of direct costs, the indirect/broader costs were not presented in all studies from societal perspective. The impact of AEFI costs into total study costs was shown to be minimal and not always easy to estimate. Summarizing and analyzing the outcomes of this study, we propose a "four steps structure" (Figure 2) that can serve as an indicator for better and more comprehensive inclusion of the AEFIs while performing economic analyses on seasonal influenza vaccine.

Interpretation
The three main reasons for not including the effects of the adverse events (AE) within the economic evaluations are referring to inconsequential differences between the compared options, minor influence on the quality-of-life, or lack of relevant data. Such approaches potentially deprive the right of decision makers and, indirectly, the target population for the vaccine intervention to be informed about the safety parameters and possible costs related to it. As the reporting of the cost-effectiveness is performed either from the payer perspective or the societal perspective, most reported costs in both perspectives account for direct medical and non-medical cost. Despite the common acceptance of their inclusion, indirect costs remain less reported than direct costs [82] also in the societal context of our reviewed studies. Such a situation is potentially imposed by the variations in national requirements regarding the pharmacoeconomic guidelines and costs or utilities to be considered [83].
It is to be expected that the serious events require longer or more complicated treatment and account for higher costs. Similarly, the local and less serious systemic AEFI would report more modest cost, as those usually included costs of over-the-count medicines. It can also happen that some of the systemic AEFIs require hospitalizations, which escalates the AEFI-related costs. What we found paradoxically in this situation, observing our reviewed studies, was having the GBS as the most commonly reported among the reviewed studies [29,34,[41][42][43]47,59,63,73,74,78], in a matter of costs, while knowing that those AEFIs are least likely to occur, 1 in 1,000,000 (vaccine associated probability) [84], or 0.8-1.9 in 100,000 person/year (population incidence) [85,86]. This AEFI was for the first time emphasized as an AEFI after receiving influenza vaccine back in the 1970s [87] and remains noteworthy until today. It might be worthwhile mentioning that all the studies that reported GBS-related costs and utilities considered the American concept, implying potential role of the population size (higher absolute number of people with AEFI) in the determination of AEFI inclusion. Furthermore, we believe that the high number of inclusions of the GBS is the result of the costly management of this severe event. However, this AEFI is not the one people will usually encounter.
such as GBS, seemed to be more interesting for inclusion in the economic evaluation on influenza vaccines than the more frequently occurring but mild AEFIs. While the reporting of AEFIs costs was preferred in a matter of direct costs, the indirect/broader costs were not presented in all studies from societal perspective. The impact of AEFI costs into total study costs was shown to be minimal and not always easy to estimate. Summarizing and analyzing the outcomes of this study, we propose a "four steps structure" (Figure 2) that can serve as an indicator for better and more comprehensive inclusion of the AEFIs while performing economic analyses on seasonal influenza vaccine.

Figure 2.
A four-step structure for inclusion of AEFI within economic evaluation on seasonal influenza vaccines. AEFIadverse event following immunization; QALYs-quality-adjusted life years.

Interpretation
The three main reasons for not including the effects of the adverse events (AE) within the economic evaluations are referring to inconsequential differences between the compared options, minor influence on the quality-of-life, or lack of relevant data. Such approaches potentially deprive the right of decision makers and, indirectly, the target population for the vaccine intervention to be informed about the safety parameters and possible costs related to it. As the reporting of the cost-effectiveness is performed either from the payer perspective or the societal perspective, most reported costs in both perspectives account for direct medical and non-medical cost. Despite the common acceptance of their inclusion, indirect costs remain less reported than direct costs [82] also in the societal context of our reviewed studies. Such a situation is potentially imposed by the variations in national requirements regarding the pharmacoeconomic guidelines and costs or utilities to be considered [83].  The utility data on AE presented in the economic evaluations can be derivates either from direct observation on patients that had the intervention, or from the literature [88]. Some literature sources from the studies in our review accounting for AEFI-related utilities led to the conclusion that the main source of the utilities is an assumption [38,40,75]. Furthermore, specifying the derivation of utilities is important, as it might occur that studies already incorporated the AEFI impact on the quality of life if the values were derived from the intervention that already accounted for AE [88]. This is particularly important if we want to have the information on specific utility, such as the AEFI-related utility, the one of interest in our research. Seen from some previous reviews, this was not the case, as they all accounted for overall QALY gains or losses [89][90][91]. With these data in mind, it should be also noted that AEFIs causing large QALY loss have the lowest probabilities of occurrence. Safety and efficacy data, as well as probabilities of occurrence and AEFI duration studies are usually combined in a trial. A survey of current practice [88] showed that the majority of the AEFIs were derived from clinical data. Each medication prior registration is subject to pre-clinical safety and efficacy trial. Such derived data are further used while performing economic evaluations on the comparing interventions. Of the same height, study-based (trial/observational/survey) economic evaluation, included in our review [35,43,44,[52][53][54]56,57,[61][62][63]65,66,68,[70][71][72]77,78,92], gave more explicit information on the AEFI than the model-based studies, which are more likely to account for parameter assumptions.
Estimating the share of AEFI-related costs into the total costs turned out to be a challenge. First, not all articles presented such data, and some even restricted their AEFI cost-data presentation by reporting it descriptively [31,44,45,60,71]. Second, we could only estimate the share of AEFI related cost that were part of the direct costs, impacting the total direct costs with less than 2% [52,56,69,73]. If this number is further used to make an estimate for the total budget, it will show an even smaller impact. This implies very modest costs of the included AEFIs in the economic evaluations on seasonal influenza. In this matter, Luce et al. [92] and Allsup et al. [93] intended to include the AEFIs in their cost analysis, but the results from the clinical trial showed no noteworthy outcome to be considered in the costs, and for that reason they excluded it from further analyses. On the other hand, Gatwood et al. reported that the economic impact of the costs of moderate AEFI is often reported, but showed considerable variation [94].
A brief review showed a scarce interest in exploring the effects of AEFIs within economic evaluations. Only one "older" survey study explored the incorporation of adverse effects in overall economic models, published between 2004 and 2007, and suggested clearer and explicit reporting [88]. Our review supports this statement, and it is the first to systematically address this issue in the context of economic evaluations on seasonal influenza vaccines. Previous reviews mainly focused on particular vaccine type or target population group, while we explored all types of seasonal influenza vaccines. Hence, de Boer et al. [91] and Thommes et al. [95] explored the effects of QIV, while Loperdo et al. [96] focused on TIV adjuvated vaccines, emphasizing the importance of age groups in selecting the vaccine type. Yet, no particular attention was brought on AEFI inclusion. Similarly, with no attention on AEFI, a review showed that children's vaccination is a cost-effective intervention [97] while emphasizing that they represent an important group in influenza transmission [98,99]. In the context of reporting on children's vaccination with seasonal influenza vaccine, our analysis showed that the economic evaluation including safety/AEFI-related parameters concerned most frequently the analysis on children. Needless to say, the recent reviews on seasonal influenza vaccine did not explored inclusion of AEFI, and therefore direct comparison to those studies was not possible. However, a review on seasonal influenza vaccines for healthcare workers reported that studies provide insufficient data to assume the effects of AEFI, emphasizing the need for their inclusion [100]. Moreover, a recent study went beyond the economic evaluation concept to investigate the public's view on vaccination strategies, including influenza vaccine, and showed that the public weight one averted AEFI equally to tree disease infections in children [101].

Strengths and Limitations
To the best of our knowledge, this is the first review on economic evaluations, mapping the AEFI-related issues on seasonal influenza vaccines, with no time, language, or target population limitation. It provides comprehensive insides of the safety-related parameters and structures them to facilitate their use into future studies. Moreover, the review followed the recommended reporting guidelines for performing a systematic review, followed the published protocol in PROSPERO, and adhered to PRISMA. Additionally, our reporting is in line with the five-step approach for conducting reviews on economic evaluations of Mastrigt et al. [83]. While other reviews on economic evaluations reported ICERs and net savings per vaccine [89][90][91], we performed a unique review on economic evaluations where the end points were the AEFI-related costs and other AEFI-outcomes, for example, health-related quality of life.
That said, the review has certain limitations. First, presenting the outcomes into meta-analysis could have brought a valuable contribution in to the future development of economic evaluations of seasonal influenza vaccines. Such analysis would have allowed us to quantify and characterize the AEFI outcomes and explore their reliability and validity. Yet, this is more common practice when performing record reviews [102,103]. Moreover, the high heterogenicity of the reported parameters, resulting from the different national requirements and policies for performing economic evaluation, did not allow such a design [100]. Therefore, all results were presented individually, and the extracted cost parameters converted in a common currency and same price year [83].
Second, the tool we used to assess the risk of bias in each economic evaluation did not include items on AEFI, or safety in general. However, we used the CHEC extended list [22,23] as it is appropriate for appraisal of trial-based and model-based economic evaluations [83], which we both included in this review. Moreover, none of the existing tools for assessing the risk of bias among the economic evaluation (such as the guidelines for authors and peer reviewers of economic submissions to the British Medical Journal (BMJ) [104], Phillips checklist [105], or the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) checklist [105]) address the AEFI issues, implying an update of these tools accordingly [106].
Third, while presenting the cost outcomes, it was not always easy to assign them a straightforward category. Those costs we assigned into wider category of other direct and other indirect costs. Moreover, the expression of unit costs was altered to the needs of the study itself and may lead to incorrect interpretation if comparing unit costs of different studies. Therefore, in our presentation, we referenced each cost. Apropos our secondary outcome, we were not able to make a share estimation of the AEFI costs in the total budget, but only in the total direct costs. This might have been the result of the frequent inclusion of AEFIs as direct costs, but also due to the fact that these costs are not being considered as highly influential in the final budgets [92,93]. For example, one study [53] presented costs for AEFI to direct costs with a value of 0.104, and in indirect cost 0.107; still, it was not clear if the costs of the hospitalization were concerning the AEFI, and thus we did not calculate their share in the total costs. Another study [63] gave values for direct (0.61) and indirect (1.47) costs of AEFI, but it was not clear what value to use for total costs since the cost for the vaccine and its administration was unknown, and there were two ways to assume this value, which would have brought us to different outcomes. In another study [62], calculated costs for AEFI turned to be 1.09% in the total vaccination (not specified direct and indirect costs) costs concerning nine patients receiving three different anti-inflammatory medications. To this end, we decided to solely include the four articles [52,56,69,73] to avoid further misinterpretation.

Research Implications
Serious AEFIs do not always occur immediately after vaccination. Follow-up within two weeks after vaccination with seasonal influenza vaccine should be sufficient to observe the common AEFI [107]. However, the serious events, such as the GBS, will not be considered within that observation time. The GBS may occur in five [87] or six [108] weeks after vaccination. Moreover, when the trial size is relatively small, we face the problem of not capturing the rare AEFI. This might be an indicator for the future performers of economic evaluations of seasonal influenza vaccine to think of the AEFI observation time as an issue that can indicate more or less costly AEFI to be considered for inclusion in the analysis, or account for real-world data on AEFI. In addition, larger-size-trials should be conducted in future in order to allow manifestation of the rare AEFI.
While the public accounts more weight to the AEFI than to the infection itself [101], within the economic evaluations, the effects of the AEFI seem to matter less. Additionally, studies often used assumptions to express the health-related quality-of-life per AEFI, showing necessity for more studies evaluating such health outcomes due to the AEFI.
Furthermore, an update of the existing guidelines for economic evaluation is needed to strengthen the requirements for inclusion of AEFI. In line with this, the tools assessing the risk of bias in the economic evaluations should also be updated to account for safetyrelated items. Moreover, we urge to emphasize the importance of coordination between vaccine and pharmacoeconomic guidelines to better utilize the AEFI reporting in economic evaluations. This will ensure a complete transparency and comprehensive analyses when safety is concerned.
In future, creation of online interactive platform for displaying categories of AEFI data, where researchers can input their AEFI-related data, or find AEFI data, would allow easy access and potently increase the inclusion of safety-related issues into economic-evaluations on seasonal influenza vaccines.

Conclusions
Our study showed that 16% of the initially eligible full-text articles considered AEFIrelated costs, utilities, frequency, or duration in the modeling, while the rest limited the inclusion to discussion only or excluded AEFIs because of assuming equality/similarity when comparing two vaccines. Direct costs, mostly from rare AEFIs, such as GBS, appeared to be the most commonly considered in economic evaluations of seasonal influenza vaccines.
Total share of its cost is minimal, but important for comprehensive preview for the decision makers as well as increased public trust in the vaccination strategies.