Fatal outcomes following immunization errors as reported to the EudraVigilance: A case series

https://doi.org/10.1016/j.vaccine.2020.02.074 0264-410X/ 2020 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). ⇑ Corresponding author at: Department of Medical Informatics, Erasmus MC, Rotterdam, the Netherlands. E-mail address: c.hoeve@erasmusmc.nl (C.E. Hoeve). 1 Authors KG and MK contributed equally to the development of this manuscript. Christina E. Hoeve a,b,⇑, Kartini Gadroen , Marcel S.G. Kwa , Anja van Haren , Miriam C.J.M. Sturkenboom, Sabine M.J.M. Straus a,b


Introduction
The introduction of vaccines has had a major impact on public health worldwide and is one of the most cost-effective medical interventions [1,2]. Vaccines are available nowadays for over 25 diseases and new vaccines continue to be developed [3]. The World Health Organisation (WHO) has estimated that 2.5 million child deaths are prevented yearly through child immunization programs [4]. The worldwide use of vaccines is large, with an estimated exposure of 100 million children yearly to diphtheria-tetanuspertussis (DTP) vaccines alone [4]. Vaccines are tested on quality, safety and efficacy before marketing authorization and are continuously monitored after marketing. Adverse events can occur following immunization some of which may be triggered by inappropriate use of vaccines. Inappropriate use of vaccines may occur due to errors in vaccine prescribing, preparation, handling, storage or administration, due to off-label use or due to misuse [5,6]. Medication errors (including immunization errors) https are defined by the European Medicines Agency as ''an unintended failure in the drug treatment process that leads to, or has the potential to lead to, harm to the patient" [6]. Examples of immunization errors are inadvertent administration of a vaccine that has been incorrectly stored or is beyond the expiry date, noncompliance with recommended vaccination schedules, using the incorrect dosage or administration of a contra-indicated vaccine; off-label use and misuse or abuse, in contrast, refer to situations where a product is intentionally used not in accordance with the terms of marketing authorization for a medical purpose (offlabel) or other purposes (misuse/abuse; e.g. recreational) [6].
Several studies have described immunization errors using passive surveillance data in the Vaccine Adverse Event Reporting System (VAERS), but none of these publications investigated immunization errors as a causative factor for fatal outcomes [7][8][9][10][11]. A study by Hibbs et al thoroughly discusses immunization errors reported to VAERS, but no details were provided on the number or causes of fatal outcomes [12]. In addition, a publication by Varricchio described immunization errors reported to VAERS between 1994 and 2001, among which one fatal case related to the use of pancuronium bromide as a diluent [13]. Very little is known about immunization errors and fatal outcome reported to the EudraVigilance database. EudraVigilance is a European database collecting individual case safety reports (ICSR) from all products licensed in the European Economic Area (EEA) [6]. We recently described 7,097 immunization errors reported to EudraVigilance between 2001 and 2016. In this study, we identified 187 ICSRs with a fatal outcome among the 7,097 ICSRs describing immunization errors [14]. In this follow-up paper, we report on an in-depth analysis of the 187 ICSRs with a fatal outcome following immunization errors as captured in EudraVigilance using the WHO tool ''Causality assessment of an Adverse Event Following Immunization (AEFI)".

Design
This study is a systematic case-series analysis of ICSRs reporting immunization errors and a fatal outcome. For all selected ICSRs the WHO tool for causality assessment of an AEFI was applied by two independent investigators. We estimated for each ICSR to what extend the immunization error might have contributed to the fatal outcome.

Data source
Data was obtained from ICSRs submitted to EudraVigilance. EudraVigilance is the European passive surveillance system for managing and analysing information on suspected adverse reactions to medicines, which have been authorised in the EEA. In this study, only post-marketing reports were considered. ICSRs are submitted to EudraVigilance by the National Competent Authorities (NCA) and product license holders i.e. Marketing Authorisation Holders (MAH), in accordance with the EU legislation. NCAs and MAHs in their turn have received reports from healthcare professionals or consumers, in addition to reports which they identify in the medical literature. All reports are subsequently followedup and coded using the Medical Dictionary for Regulatory Activities (MedDRA Ò ) terminology 2 . Although follow-up is a legal requirement, this may not always be feasible and practices vary on the circumstances (e.g. whether the reporter is a health care professional and/or whether the patient has given consent for consulting medical record, etc). Reporting requirements for the submission of ICSRs are strictly regulated [15]. Criteria for a valid case report and definitions of the data elements are specified in the International Conference on Harmonization guidelines [16]. Case reports published in the medical literature may qualify for submission to EudraVigilance if they fulfil the criteria for a valid case report and the product involved is licensed in the EEA [6]. Standardized MedDRA Queries (SMQs) have been developed by the 'Council for International Organizations of Medical Sciences (CIOMS) working group for SMQs' to extract certain types of events [17].

Data source and collection
All post-marketing ICSRs submitted to EudraVigilance within the study period (1 January 2001 through 31 December 2016) with the following characteristics were included: (1) reporting vaccines as a suspected product, (2) having a medication error term and (3) death as outcome. Submission date was determined by the ICSR receive date which corresponds with the date when an NCA or MAH received the initial report from the reporter. In case there are multiple versions of the same report, only the latest version was included. Medication errors were extracted using the standard narrow SMQ: a collection of MedDRA preferred terms (PTs) to identify medication errors cases e.g. 'administration error' or 'product administered to wrong patient'. All PT's listed in the narrow SMQ are provided in the supplementary table A. Reported exposure to vaccines was determined by using search terms such as human papilloma, herpes, influenza, et cetera (a full list of all terms is provided in supplementary table B).
The following data was collected from the reports (if available): Patient's age and gender, geographical location (EEA vs non-EEA), primary source qualification (healthcare professional or nonhealthcare professional), vaccine(s) involved in the error and the type of error committed with vaccine (MedDRA PT). If the information in a case report was limited and there was a reference to a publication included, the publication was retrieved for additional information. In case a reference was included, the appropriate publication is referenced when the cases are described below. Country economy status was based on the most recent categorisation by the World Bank: lower-income economies (Gross National Income (GNI) per capita of $1,025 or less in 2018), lower-middle-income economies (a GNI per capita of $1,026-$3,995), upper-middleincome economies (GNI per capita of $3,996-$12,375), and high income economies (GNI per capita >$12,376). [18].

Causality assessment
Two factors are of relevance when considering a potential causal association between an immunization error and a fatal outcome: (1) the contribution of the vaccine to the fatal outcome, and (2) the contribution of the error to the fatal outcome. First, the contribution of the vaccine was determined. The ICSRs were assessed by two independent reviewers (MK, CH) using the WHO tool ''Causality assessment of an Adverse Event Following Immunization (AEFI)" to determine the level of certainty of a causal association between the exposure (immunization error) and event (death) [19]. The WHO AEFI causality tool uses a four-step approach as follows, in summary:

Step 1. Eligibility
In this step it is determined whether the case report satisfies the minimum criteria for causality assessment. First, it should be confirmed that the vaccine was administered before the occurrence of the reported event. Second, a diagnosis for the reported AEFI should be present (in this case the underlying cause of death). 2 MedDRA Ò is the international medicinal terminology developed under the auspices of the International Council on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use.

Step 2. Checklist
Eligible cases are reviewed using the checklist (see supplementary file), which takes multiple factors into consideration: evidence of other causes, a known causal association between the vaccine and reported event, the time to onset of the event and the presence of strong evidence against a causal association [19].

Step 3. Algorithm
An algorithm is applied to the data identified in the checklist (see supplementary file) while taking into account expert knowledge and logical deductions. This algorithm classifies the causality of AEFI in four main categories and eight sub-categories [19]. The first three main categories are defined as consistent causality, indeterminate, or inconsistent causality (Table 1). Although cases may be classified as eligible in step 1, if information is lacking for full assessment after completion of the checklist in step 2, these cases are categorized as ''unclassifiable".

Step 4. Classification
Causality is further classified according to the 7 sub-categories presented in Table 1, providing more detail on the type of causal association.

Assessment of the role of 'error'
In addition to the causality assessment based on the WHO tool, we tried to ascertain to what extent the error might have contributed to the outcome and ranked the cases accordingly. We performed an assessment on each case, for which we first assessed the possible causality of the vaccine (regardless of the error) in relation to the fatal outcome as described above. After categorization of the causality, an estimation was made by both reviewers of the potential contribution of the error on the outcome. The reviewers' main question to be assessed was: would death have occurred if the error had not occurred? Based on the reviewers' personal assessment the error contribution for each case was categorized as large, moderate, small, unclassifiable or none ( Table 2). Some examples of error contribution assessment are provided in the boxes.

Example case A:
An immunocompromised patient dies because of a vaccine Astrain infection after immunization with live-attenuated vaccine A, whereas the patient was supposed to have received vaccine B (not live). Therefore, if vaccination had been administered according to plan, the patient would never have encountered the vaccine A-strain infection that became fatal. Estimated error contribution: large.

Example case B:
A patient is supposed to receive a vaccine with the following schedule: day 0, 7 and 30; but instead receives it only at day 0 and 7. Three months later the patient dies of infection with the virus the vaccine was supposed to prevent. It cannot be excluded that the patient would not have died of the infection if he/she had received the complete series of the vaccine, but it is likely. Estimated error contribution: moderate.

Interrater agreement
Cases where differences existed in the assessment of causality or error contribution categories were discussed to reach consensus. A kappa coefficient was calculated to determine the inter-rater agreement both for causality assessment as well as error contribution.

Description of selected ICSRs
In total 187 ICSRs were identified reporting immunization errors with fatal outcome in the study period. After initial evaluation of the cases 32 ICSRs were excluded for reasons as shown in Fig. 1. The remaining 154 ICSRs were mostly reported by health care professionals (91.6%) and originated most frequently from non-EEA countries (82.5%). In addition, the majority of ICSRs was reported from high-income countries (63.6%) ( Table 3). Fifty-four ICSRs (35%) had publications in the medical literature, local newspapers or regulatory magazines. 58 (37.7%) ICSRs concerned females, 74 (48.1%) ICSRs males and in the remaining ICSRs gender was not specified ( Table 3). The most frequently implicated vaccines were pneumococcal vaccines (Table 4), whereas the most frequently reported immunization error was non-compliance to the recommended vaccination schedule ( Table 5). The combination of error and vaccine that was reported most frequently was rabies vaccines with non-compliance with recommended post-exposure prophylaxes ( Table 6). An overview of the causality and error contribution is provided for each ICSR in the supplementary material reporting vaccines, error terms, age category and gender of the patients (supplementary table).

Causality assessment classification
3.2.1. Cases consistent with causal association to immunization and large error contribution (n = 12) For a total of 12 ICSRs (7.8%) the immunization was classified to have a causal association with the fatal outcome and the immu- Table 1 Categories specified by the WHO tool for causality assessment for AEFI [18). 1. There is insufficient information provided in the case report to allow for assessment of causality Table 2 Categories for estimation of the extent to which errors have contributed to fatal outcomes.

Category Subcategories
Large The information supports that chance that the error contributed to the fatal outcome was large.

Moderate
There is a chance that the error contributed to the fatal outcome, but there are also other factors present that could have contributed to the fatal outcome. Small The information supports that the chance that error contributed to the fatal outcome was small. There are other more likely factors. Unclassifiable There is insufficient information provided in the case report to allow for assessment of the error contribution. None No error could be identified in the case narrative (e.g. offlabel use).
nization error had a large contribution to the outcome (Fig. 1). Six of these ICSRs were based on one single publication that suggested that the fatal outcomes were probably caused by the administra-tion of contaminated reconstituted multi-dose vial of measles vaccine [20]. The root cause was incorrect handling of the multi-dose vial after opening of the vial. The error contribution according to the reviewers was large, since the children might not have died if the error had not occurred. The other six ICSRs concerned the administration of live-attenuated vaccines to immunocompromised patients: three cases with live-attenuated varicella vaccines in patients with immune suppression or chemotherapy [21,22], two cases with Bacillus Calmette-Guérin (BCG) vaccines in patients with immunodeficiency disorders (STAT1 deficiency and IL7RA), and one case with a live-attenuated rotavirus vaccine in a patient with Severe Combined Immune Deficiency (SCID) [23]. All six patients developed fatal disseminated infections; in five cases the vaccine strain was detected in the patient. In most cases it was unclear whether the immune status of the patient was known at the moment of immunization; In one case the patient completed immunosuppression therapy 6 months before immunization and was therefore considered eligible to receive varicella according to the local recommendations at that time [24].

3.2.2.
Cases consistent with causal association to immunization and moderate or lower error contribution (n = 8) In eight ICSRs (5.2%) the vaccine was assessed as the causal agent for the fatal outcome and, according to the reviewers, the error contribution was assessed as moderate or less (Fig. 1). These cases concerned a variety of vaccines: poliomyelitis vaccine [25], varicella vaccine [26], influenza vaccine [27], rotavirus vaccine, HPV vaccine [28], yellow fever vaccine [29], measles, mumps, and rubella (MMR) vaccine [30] and Influenza + varicella + MMR vaccine.  ICSRs the vaccine did not belong to the J07 ATC group (these concerned treatment with BCG vaccines for primary or recurrent carcinoma in situ (CIS) of the urinary bladder), in 3 ICSRs the error which occurred did not concern a vaccine, but another co-reported medicine; and in 1 ICSR there was no error. The outcome of assessment of vaccine causality and error contribution of the remaining 154 ICSRs is illustrated.
Two of the eight cases were classified as error contribution moderate: in one case the patient (South-America, <2010) was diagnosed with vaccine-associated paralytic poliomyelitis (VAPP) 40 days after receiving the oral poliomyelitis vaccine at the age of 15 months instead of the recommended age of two months. It was not fully established that the poliomyelitis was caused by the vaccine strain; therefore, delayed administration may also have led to reduced protection and infection with a wild strain. The latter scenario is however quite unlikely considering that indigenous wild poliomyelitis virus transmission has been eradicated from the Americas for two decades [31]. In the second case a disseminated infection with a varicella strain was identified in the patient after accidental administration, but the cause of death was reported as MRSA infection. It is not certain that the accidental administration of varicella vaccine to the immunocompromised patient was the cause of death.
Three of the eight cases were classified as error contribution small. One case concerned administration of an influenza vaccine type A and B to a patient aged 3.5 years, who within hours following immunization developed an allergic reaction which was fatal despite corrective treatment. The vaccine is therefore considered causally related. The immunization error was reported as 'inappropriate age of vaccination' However, in this case, it is not likely that the age of the patient has contributed to this event. In the second case a patient received a live-attenuated rotavirus vaccine at age three months instead of the recommended two months. The patient passed away following complications due to intussusception. Administration 1.5 months later than the recommended age may have contributed to the development of intussusception since this risk increases with increasing age of immunization [32]. In the third case it was reported that a patient received two doses of HPV vaccine non-compliant with the recommended schedule (exact time of second administration not reported). The patient developed systemic lupus erythematosus, a reaction that has also been reported in the literature in association with HPV vaccination [33]. The error contribution was assessed as unlikely in this case.
Three cases were considered to be incorrectly reported as error or the error was unclassifiable: one case concerned intramuscular administration of yellow-fever vaccine which is compliant to the EU-SmPC recommendation (although subcutaneous administration is preferred). The patient developed yellow fever vaccine-associated viscerotropic disease and in line with the WHO tool the case was classified as consistent causality. The second case reported liveattenuated MMR vaccine administration to an immunocompromised child. This was not considered an error since the immunodeficiency was not diagnosed until after immunization. However, the patient developed vaccine-associated mumps encephalitis which lead to the fatal outcome, hence the causality was consistent. In the remaining case a patient was vaccinated at age 2.5 years with influenza, varicella and MMR vaccines which was reported as inappropriate age. The patient developed an anaphylactic reaction with fatal outcome. Causality with immunization is likely, however, it cannot be concluded which vaccine caused the event. In this case the contribution of the error could not be classified.

Cases inconsistent with causal association to immunization and large error contribution (n = 25)
In a total of 25 ICSRs (16.2%) the contribution of the error in the fatal outcome was assessed as large; Regardless of the reported error, the vaccine was not considered to have a consistent causality with the fatal outcome (Fig. 1). Twenty of the 25 cases concerned errors with post-exposure prophylaxis (PEP) with rabies vaccines. Sixteen of the 20 cases described non-compliance to the recommended immunization schedule or incomplete course of immunization (supplementary table) and 15 of the 20 cases also reported incorrect administration of immunoglobulins. Noncompliance to the recommended schedule of rabies PEP concerned delayed onset (n = 6) or incomplete/wrong pattern of vaccine administrations (n = 11). Reasons for non-compliance to the schedule were of varying nature: the patient was not aware of urgent need of immediate treatment, the patient could not afford vaccines, the patient ignored doses following primary vaccination, shortages at the treatment center or the patient fell ill before completion of the schedule. Incorrect immunoglobulin administration concerned delayed administration (n = 1), administration not in wound or only half in wound (n = 5), underdose (n = 1), or RIG was not given (n = 8). In all 20 cases the patient died from rabies infection. The majority of cases was reported in middle-income countries [17], with two cases from a high-income country (non-EEA) and one case from a low-income country. Thirteen of the 20 cases were based on publications in the medical literature [34][35][36][37][38][39][40][41][42][43][44]. Other cases with large error contribution but non-consistent causality with the vaccine concerned confusion of hepatitis B vaccines with insulin (n = 1) and rocuronium bromide (n = 1). In both cases the wrong drug was accidentally administered due to confusion of packaging similar to the hepatitis B vaccine packaging. One other case described incomplete vaccination schedule with a pneumococcal vaccine. The patient was insufficiently immunized and developed fatal pneumococcal meningitis. One other case described a patient who missed a booster dose of diphtheria, tetanus, pertussis, and Haemophilus influenzae type B (DTP-Hib) vaccine, which may have caused insufficient immunity and subsequently developed fatal diphtheria. The remaining case described dilution of a BCG vaccine with a neuromuscular blocker leading to death.
3.2.4. Cases classified as indeterminate or inconsistent with causal association to immunization and moderate, small, unclassifiable error contribution or no error reported (n = 76) Vaccines reported most frequently in these categories were pneumococcal vaccines (n = 24), followed by influenza vaccines  (n = 14), and bacterial and viral vaccines combined e.g. Diphteria + Tetanus + Pertussis + Hepatitis B + Poliomyelitis + HIB (n = 9) (supplementary table C). The most frequently reported error was noncompliance to the recommended immunization schedule [28] (supplementary table C). No clear pattern of particular errors for specific vaccines was observed in these categories. For the cases with indeterminate or inconsistent causality either an alternative cause of death was confirmed, or the cases were confounded by underlying conditions or concomitant administration of other medication. Eight of the 76 cases reported sudden infant death syndrome (SIDS).
3.2.5. Unclassifiable cases (n = 54) or cases reporting no error [15] Due to a lack of information in the reports for a total of 54 cases the causal association between the vaccine and fatal outcome and/ or the error contribution was unclassifiable (Fig. 1). In 31 of the 54 cases neither causality nor error contribution could be assessed. In 45 of the 54 cases there was no information on the cause of death or it was not detailed enough for assessment of the causality (e.g. viral infection, but no information on the origin of the strain). In 24 cases information on the errors was not detailed enough, e.g. when inappropriate age or schedule of vaccine administration was reported but no details were provided on the age or appropriate schedule. Other information often lacking was data on the medical history (n = 34), concomitant medication (n = 33), time to onset (n = 24), or no narrative was provided (n = 5). In 15 of the 154 cases review of the case narrative pointed out that there was no actual error in the treatment (e.g. off-label use), in addition to the 1 case without error which was identified before review.

Discussion
Our main finding is that in the majority of the 154 reported fatal cases of immunization errors to the Eudravigilance database, identified within a 16-year period, there was no sufficient ground to confirm a possible causal association between immunization error and fatal outcome. In a third (n = 45) of the cases, the contribution of the error as assessed by the investigators was classified as small. We showed that reported fatal outcomes following immunization errors are very rare in view of the large worldwide exposure to vaccines. In addition, it should be viewed in the context of 2.5 million deaths prevented yearly by childhood vaccination [4].
Four key lessons could be identified from our analysis. First, the reports discussing the administration of contaminated vaccines stress the need for continuous control of vaccines and their diluents before immunization. Administration of expired vaccines or inappropriately stored vaccines may affect the safety and effectiveness of vaccines [45]. As a result, the patient may be insufficiently protected against the disease for which the vaccine is intended. It may not only have an impact on the individual patients but can also affect public trust in immunization programs. For example, following administration of a contaminated measles vaccine in India in 2008, local immunization programs were suspended due to fear in the population, even though there was no proper investigation into the cause of the fatal events [46,47]. Innovative mea-sures could be helpful to support correct handling of liveattenuated multi-dose vaccines and to ensure that vials are discarded within the recommended time interval after opening, e.g. change of colour of the vaccine solution after opening in line with vaccine vial monitors for maintenance of the cold-chain [48].
Second, almost half of the cases rated with consistent causality reported administration of live-attenuated vaccines to immunocompromised patients. Intentional administration of a liveattenuated vaccine to an immunocompromised patient is considered off-label. Fatal outcome is a known risk with live-attenuated zoster vaccination in immunocompromised patients [49][50][51][52]. Even though it remains unclear whether the vaccines were administered erroneously or as (intentional) off-label use, these cases show the need for careful evaluation and checking of the immune status of the patient before immunization with live vaccines. Furthermore, differences in accepted time between immunocompromising treatments and immunization in different guidelines calls for more research regarding the duration of immune-suppressive effects following cessation of therapy.
Third, almost half of the cases where large error contribution was identified concerned non-compliance with recommended schedules or an incomplete series of vaccination. Many of these cases related to rabies PEP. Incomplete rabies PEP is a global and recognized problem [34][35][36][37][38][39][40][41][42][43][44]. In some cases it was also identified that administration of immunoglobulins in the wounds was not performed according to recommendations. In some cases this was due to lack of access to rabies vaccine and/or immunoglobulin in low and middle-income countries. When information on the cause of incomplete rabies PEP was available, it showed that the reasons varied, ranging from inadequate knowledge with the patient on the seriousness of animal bites, the need for urgent treatment, high costs of treatment, or shortages at treatment centres. Educating people, to increase knowledge on the potential seriousness of an animal bite and the required speed of treatment are needed and may improve rabies PEP treatment. In addition, the costs of treatment should be affordable to all in need and sufficient stock should be available in treatment centers located in rabies endemic areas. Non-compliance to the recommended immunization schedule was also reported for pneumococcal vaccines. Studies have been published describing low adherence to pneumococcal immunization schedules in children [53,54]. The root cause of these errors could not be identified from our data; therefore, no clear-cut recommendations can be made based on these cases.
Fourth, in two cases it was reported that hepatitis B vaccines were confused with other medication due to similar packaging. In the EU clear guidelines are in place to limit similarity in packaging [55]. Furthermore, the WHO has released recommendations to limit confusion between vaccines and medicines by e.g. separate storage places for these products. Our findings support these recommendations.
A few of the evaluated cases reported SIDS (n = 8). Cases are considered SIDS when an apparently healthy infant of <1-yearold dies from an unexpected death with no explanation after thorough investigation and autopsy. Considering the large exposure of the world population to vaccines and the rate of infant death, it is expected that deaths occur also closely after vaccine administration, without it being caused by the vaccine. Various studies have investigated death following immunization using observational data, especially in the pediatric population [56][57][58][59][60][61]. These studies have all ruled out a causal association with vaccine administration and death, both in cases of SIDS and also unspecified death [56][57][58][59][60][61][62][63][64]." Over a third of the cases of immunization errors with fatal outcome in Eudravigilance have been published in the medical literature. It is important that information on immunization errors and fatal outcome is in the public domain. This can help health institutes, regulators and industry to take lessons learnt from mistakes and develop measures for prevention of these errors. Although occasionally case reports of fatal immunization errors in lowand middle-income countries (LMIC) are published in the medical literature or media, we observed that a majority of our cases originated from higher-income countries (HIC). These numbers are in not in line with the exposure. Current estimates show that the number of vaccine doses administered in LMIC is higher than in HIC [65]. In a large number of countries AEFI reporting is still very low [66] and underreporting may be higher in LMIC. Reasons for the low reporting rate may be low detection and reporting of AEFI cases, low quality of data and lack of clarity in roles and responsibilities in LMIC [67]. When individual cases are reported through the media, these cases often do not full-fill the criteria for reporting in EudraVigilance and may therefore be lacking [6]. AEFI reporting in LMIC is of importance, especially considering the fact that new vaccines are in some cases mainly used in these countries (e.g. Mosquirix (authorized through EMA) or rVSV-ZEBOV (currently under review for approval by EMA)). For these vaccines limited knowledge on safety and efficacy in a large population is available, therefore appropriate surveillance is required.

Limitations
We used a publicly available tool, the WHO causality tool, to review the cases. Nevertheless, we observed a moderate interrater agreement. Although a few studies have been published using this WHO tool for causality assessment, the tool has not been tested for reliability, reproducibility and consistency [68]. Others who used the tool (e.g. Puliyel et al) reported similar issues we observed [69]. First, when using the WHO tool, in ICSRs reporting immunization errors the errors are directly assumed causally related to each of the reported AEFI. The manual states that the focus for errors is on the error and not on the underlying biological processes. In addition, the tool does not provide possibilities to identify the factor causing the AEFI (vaccine or error), nor the root cause of the error which is necessary for developing appropriate tools to minimize the risk of error during future use with the product. The outcomes of our study show that an error is not necessarily the cause of a reported AEFI. Second, the tool allows for ICSRs to be classified in multiple categories simultaneously (i.e. a single case can be classified both consistent with causal association and also inconsistent with causal association). Final selection of the appropriate classification is based on expert opinion of the reviewer, allowing for varying interpretation of data. Third, the tool requires a valid diagnosis for each AEFI to be assessed to render an ICSR eligible for further evaluation. This step is illogical, as a diagnosis is usually only decided on later in the assessment. In our analysis limited information on the diagnosis often led to an unclassifiable case. Another limitation of this study is that ICSRs are collected from Eudravigilance. Assessing causality can be challenging using spontaneous reporting data due to the well-known limitations of these data e.g. limited quality, incomplete data, underreporting, and reporting bias [70][71][72]. The latter is especially relevant for events like death, as this event may much sooner be reported than less serious events, even if coincidental [70]. These are common issues with spontaneous adverse event reporting data and the primary goal of these spontaneous reporting databases is to detect signals and generate hypothesis, rather than confirm potential vaccine event combinations. Immunization errors which were not coded as such by the MAH or NCA are missed in this study.

Conclusion
This study describes the review and assessment of a case series of ICSRs reporting fatal outcomes following immunization errors. We showed that reported fatal outcomes following immunization errors are very rare in view of the large worldwide exposure to vaccines. Four main lessons can be drawn: (1) the continuing need for quality control during administration of multi-dose vaccines, (2) the importance of screening of patients for immunocompromising factors when a live-attenuated vaccine is used, (3) the need to educate and instruct health care professionals and patients on the importance of adherence to vaccination schedules (especially rabies PEP), and (4) the importance of measures to improve visually distinguishing features on vaccines and medicines.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.