A safety study evaluating non-COVID-19 mortality risk following COVID-19 vaccination

Background The safety of COVID-19 vaccines plays an important role in addressing vaccine hesitancy. We conducted a large cohort study to evaluate the risk of non-COVID-19 mortality after COVID-19 vaccination while adjusting for confounders including individual-level demographics, clinical risk factors, health care utilization, and community-level socioeconomic risk factors. Methods The retrospective cohort study consisted of members from seven Vaccine Safety Datalink sites from December 14, 2020 through August 31, 2021. We conducted three separate analyses for each of the three COVID-19 vaccines used in the US. Crude non-COVID-19 mortality rates were reported by vaccine type, age, sex, and race/ethnicity. The counting process model for survival analyses was used to analyze non-COVID-19 mortality where a new observation period began when the vaccination status changed upon receipt of the first dose and the second dose. We used calendar time as the basic time scale in survival analyses to implicitly adjust for season and other temporal trend factors. A propensity score approach was used to adjust for the potential imbalance in confounders between the vaccinated and comparison groups. Results For each vaccine type and across age, sex, and race/ethnicity groups, crude non-COVID-19 mortality rates among COVID-19 vaccinees were lower than those among comparators. After adjusting for confounders with the propensity score approach, the adjusted hazard ratios (aHRs) were 0.46 (95% confidence interval [CI], 0.44–0.49) after dose 1 and 0.48 (95% CI, 0.46–0.50) after dose 2 of the BNT162b2 vaccine, 0.41 (95% CI, 0.39–0.44) after dose 1 and 0.38 (95% CI, 0.37–0.40) after dose 2 of the mRNA-1273 vaccine, and 0.55 (95% CI, 0.51–0.59) after receipt of Ad26.COV2.S. Conclusion While residual confounding bias remained after adjusting for several individual-level and community-level risk factors, no increased risk was found for non-COVID-19 mortality among recipients of three COVID-19 vaccines used in the US.


Introduction
Four COVID-19 vaccines have been authorized in the United States since December 14, 2020. The two mRNA COVID-19 vaccines, BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna), have been widely used while the adenoviral vector vaccine, Ad26.COV2.S (Janssen), has been available but used more sparingly compared to the mRNA vaccines. NVX-CoV2373 (Novavax) was authorized in the United States in July 2022, after the study period.
Several studies have examined mortality risk after COVID-19 vaccination, although they had limited sample size, were restricted to specialized populations (e.g., nursing home residents), lacked a comparison group, or did not comprehensively adjust for confounders. A moderate-sized cohort study of 21,222 nursing home residents compared all-cause mortality between COVID-19 mRNA vaccinees and unvaccinated residents and found that vaccinees had lower all-cause mortality after adjusting for some confounders. [15] A longitudinal study compared mortality rates over time among vaccinated patients in the U.S. Veterans Affairs health system with no history of COVID-19 and found no evidence of excess mortality associated with receipt of mRNA vaccines. [16] Preliminary results in a large cohort study showed that COVID-19 vaccine recipients had lower rates of non-COVID-19 mortality than did unvaccinated comparators after adjusting for age, sex, race/ethnicity, and study site, [17] suggesting possible effects of unmeasured confounders and healthy vaccinee effects (i.e., vaccinated persons tend to be healthier than unvaccinated persons). [18,19].
This study aimed to evaluate the risk of non-COVID-19 mortality after COVID-19 vaccination in a large cohort of individuals using survival analyses and an improved inverse probability of treatment weighting (IPTW) approach to adjust for confounders including individual-level demographics, clinical risk factors, health care utilization, and community-level socioeconomic risk factors. We hypothesized that COVID-19 vaccines do not increase the risk for non-COVID-19 mortality despite their association with some rare severe adverse events.

Study design and population
We conducted a retrospective cohort study among health plan members aged ! 12 years enrolled in seven Vaccine Safety Datalink (VSD) sites (Kaiser Permanente [KP] Southern California, KP Northern California, KP Colorado, KP Northwest, KP Washington, HealthPartners, and Marshfield Clinic). The VSD population is socio-economically diverse and represents about 3% of the U.S. population. [20] Vaccination status was assessed from December 14, 2020 through June 30, 2021, and deaths were assessed until August 31, 2021 to allow at least two months of follow-up. Follow-up was censored upon any COVID-19 vaccination between July 1, 2021 and August 31, 2021.

Exposure
The exposure was vaccination with one of three authorized COVID-19 vaccines: BNT162b2, mRNA-1273, and Ad26.COV2.S. Three separate analyses were conducted for each of the three vaccines with separate comparator groups. We performed weekly frequency matching on age and sex within each VSD site. [17] For a given week and a pre-specified matching ratio, COVID-19 vaccine recipients of dose 1 during the week were identified and their vaccination dates were used to assign index dates to comparators who had not been vaccinated as of that date and were randomly selected according to the matching ratio. We allowed those comparators who were matched in a previous week to switch to being vaccinated upon receiving a COVID-19 vaccine. The matched ''comparators" thus included both pre-vaccination person-time among   COVID-19 vaccinees as well as unvaccinated person-time of individuals who did not receive any COVID-19 vaccines by June 30,  2021. For the mRNA vaccines, individuals who received the vaccines from December 14, 2020 through June 30, 2021 were included in the vaccinated group. The weekly frequency matching ratio of vaccinated individuals and comparators was about 1:1. Exposure had three levels: pre-vaccination, after dose 1 and after dose 2. For Ad26.COV2.S recipients, individuals who received the vaccine from February 27, 2021 through June 30, 2021 were included in the vaccinated group, and the matching ratio was 1:4. Exposure had two levels: pre-vaccination and after dose 1.
Individuals were followed until death, disenrollment, receipt of a COVID-19 vaccine for unvaccinated comparators, or the end of follow-up (August 31, 2021), whichever occurred first. When individuals received different vaccine products for dose 1 versus dose 2, their follow-up was censored upon receipt of the second mismatched dose. To be included in this study, individuals were required to have ! 1 year enrollment in the health system before their index dates for their confounders to be properly measured. To increase comparability of health care-seeking behavior between COVID-19 vaccinated and unvaccinated individuals, we required that comparators had received ! 1 dose of influenza vaccine in the two years prior to the index date.

Outcomes
Since this was a safety study of COVID-19 vaccines, the primary outcome was non-COVID-19-associated death during follow-up, as COVID-19 vaccination was expected to be protective against COVID-19-associated death. We first identified deaths through VSD data files capturing hospital deaths and deaths reported to health plans, and then excluded deaths occurring 30 days following a COVID-19 diagnosis or a positive SARS-CoV-2 test. Secondary outcomes included 30-day non-COVID-19 mortality in which follow-up was censored 30 days after the index date, and allcause mortality which included deaths from all causes including COVID-19.

Confounders
We considered individual-level confounders including age, sex, race/ethnicity, Medicaid status, history of COVID-19, number of combined outpatient and virtual visits within one year prior to the index date, inpatient visit (yes/no) within one year prior to the index date, Emergency Department (ED) visit (yes/no) within one year prior to the index date, inpatient or ED visit within 7 days prior to the index date (yes/no), presence of frailty measured within one year prior to the index date (yes if frailty index ! 0.11; no, otherwise), [21] Charlson Comorbidity Index (CCI) within one year prior to the index date, receipt of another vaccine within 14 days before or after the index date, neighborhood median household income, and neighborhood education level. Healthcare Common Procedure Coding System (HCPCS) codes that were used in the development of frailty scores were not available in this study, resulting in lower frailty scores. Therefore, we chose a frailty score of 0.11 as the cut-off for the presence of frailty. Neighborhood-level education was defined as < 50% or ! 50% of the neighborhood attaining > high school education.

Statistical analyses
For each vaccine type and dose and comparator group, crude non-COVID-19 mortality rates per 100 person-years were calculated as (number of non-COVID-19 deaths/person-years) Â 100.
To reduce confounding bias in this observational study, we employed a propensity score weighting approach to adjust for the potential imbalance in confounders between the vaccinated and the comparison groups. [22,23] Separate propensity score models were created for the three vaccine cohorts. For the two mRNA vaccines, we fit a multinomial model because the dependent variable in the propensity score model, COVID-19 vaccination, had three levels. [24] For Ad26.COV2.S, we fit a logistic regression model because the dependent variable, COVID-19 vaccination, had two levels. Based on the propensity score models we calculated stabilized weights (SW), [25] an improved inverse probability weighting approach in survival analyses. SWs not only reduce the impact of some extreme weights but also preserve the original sample size. [26] Balance in measured confounders between vaccinated and comparison groups was assessed with absolute standardized mean differences (SMD) before and after applying SWs. An absolute standardized mean difference of <0.10 indicated good confounder balance. [27].
The counting process model for survival analyses was used. A new observation period began when the vaccination status changed upon receipt of the first dose and the second dose. [28,29] We used calendar time as the basic time scale in survival analyses to implicitly adjust for season and other temporal trend factors. [30] We estimated both unadjusted and SW-adjusted hazard ratios (aHR) and 95% confidence intervals (CI) of vaccination effects on non-COVID-19 mortality, 30-day non-COVID-19 mortality, and all-cause mortality.
To detect possible bias from inadequate confounding adjustment, we also conducted exploratory negative control outcome analyses [31] separately for each of the three COVID-19 vaccines in which we replaced the outcome of death with first occurrence of trauma or injury hospitalization during the exposure followup period (i.e., vaccinated or unvaccinated). We hypothesize that the negative control outcome, hospitalization for trauma or injury, shares the same potential sources of bias with our primary outcome (death) but cannot plausibly be related to COVID-19 vaccination. [18,32] Trauma or injury hospitalizations were identified with the following ICD-10 codes: S00-T88 for injury, poisoning and certain other consequences of external causes, and V00-Y99 for external causes of morbidity. [33] A similar analytic approach as for the primary outcome (death) was used in the negative control outcome analyses. SWs were estimated from propensity score models where the same covariates for the primary outcome were included, and the receipt of COVID-19 vaccination was the dependent variable. We analyzed time since the calendar date of receiving the first dose among vaccinees or the corresponding index date among comparators to an incident trauma or injury hospitalization during the exposure follow-up period with and without applying SWs.

Characteristics of COVID-19 vaccine recipients and their comparators
In total, 6,974,817 unique individuals (vaccinated and unvaccinated) were included in the study, with 5,107,262 unique individuals for analyses of BNT162b2, 4,037,724 unique individuals for analyses of mRNA-1273, and 1,510,652 unique individuals for analyses of Ad26.COV2.S. Some comparators appeared in more than one analytic cohort. By June 30, 2021, 3.3 million individuals in the study received at least one dose of BNT162b2, and 93.4% of them received two doses (Table 1); 2.4 million individuals received at least one dose of mRNA-1273, and 95.0% of them received two doses ( Table 2). There were 331,282 individuals who received Ad26.COV2.S by June 30, 2021 (Table 3). Across vaccine types and doses, vaccine recipients and their comparator groups were comparable, with a few minor differences between groups (SMD greater than 0.10). However, application of SWs to the cohorts reduced the absolute SMD for all confounders to below 0.01 (Fig. 1).
The composition, sample sizes, and person-years of the study population are presented in Supplemental Table 1 after allowing unvaccinated comparators to switch to being vaccinated upon receiving a COVID-19 vaccine. Compared to vaccinated individuals, the average of follow-up among comparators was shorter mainly due to censoring upon receipt of a COVID-19 vaccine. The ratios of sample size of those who were ever vaccinated to those never vaccinated as of June 30, 2021 were 3,281,777: 902,814 = 1:0.28 for BNT162b2, 2,393,784: 676,955 = 1:0.28 for mRNA-1273, and 331,282: 523,615 = 1:1.6 for Ad26.COV2.S.

Crude mortality rates
Across vaccine types and doses, the crude non-COVID-19 mortality rates in vaccine recipients were lower than those in the corresponding comparator group. For BNT162b2, the crude non-COVID-19 mortality rates were 0.76 and 0.66 per 100 personyears for dose 1 and dose 2, respectively, while the comparator group had a crude mortality rate of 1.76 per 100 person-years (Table 4). For mRNA-1273, the crude non-COVID-19 mortality rates were 0.76 and 0.67 per 100 person-years for dose 1 and dose 2, respectively, versus 2.04 in the comparator group (Table 5). Ad26.COV2.S recipients had a crude mortality rate of 0.82 per 100 person-years, versus 1.58 in the comparator group (Table 6).
Across vaccine types and doses, aHRs of 30-day non-COVID-19 mortality and of all-cause mortality were lower than those from the analyses of non-COVID-19 mortality (Table 7).

Discussion
In this study of more than 6 million recipients of COVID-19 vaccines and their unvaccinated comparators, we found that recipients of BNT162b2, mRNA-1273, and Ad26.COV2.S vaccines had lower non-COVID-19 mortality risk than their comparator groups. For mRNA vaccines, the aHRs of dose 1 and dose 2 ranged from 0.38 to 0.48. These primary analysis findings of no increased mortality risk among COVID-19 vaccine recipients are consistent with existing knowledge about mortality risk after COVID-19 vaccination. [15][16][17] The aHRs of all-cause mortality were lower than those from the analyses of non-COVID-19 mortality, likely due to the protection of COVID-19 vaccines against COVID-19 infection, severe illness, and deaths. The findings suggested some all-cause mortality benefit of COVID-19 vaccines for unknown causes in addition to their known protection against COVID-19 infection, severity of the disease and death. While previous studies have suggested that live attenuated vaccines may be associated with lower risk of nonvaccine-targeted infections, [34][35][36] it is unclear whether trained immunity might also be induced by mRNA and adenoviral vector COVID-19 vaccines. If so, such non-specific protection against heterologous infection could lead to decreased mortality due to non-COVID-19 causes.
A recent study in Hungary demonstrated the effectiveness of COVID-19 vaccination in reducing all-cause mortality after adjusting for measured confounders and potential healthy vaccinee effect when compared to unvaccinated individuals. [37] A VSD study found that the mortality rates were lower in the days immediately following vaccination in a cohort of adults and children between January 1, 2005 and December 31, 2008, indicating a healthy vaccinee effect. [38] Another VSD study included individuals aged 9 to 26 years with deaths between January 1, 2005 and December 31, 2011. A case-centered method was used to estimate a relative risk (RR) for death in days 0 to 30 after vaccination. It was shown that RRs after any vaccination and influenza vaccination were significantly lower for deaths due to nonexternal causes and all causes.
The authors suggested that vaccination would be less probable in individuals whose death was imminent. Also, since the population was relatively unhealthy, this bias might not be from the traditional healthy vaccinee effect, but rather from unmeasured confounding related to the timing of vaccination by indication or disease severity. [39]. Jackson et al [18] used trauma or injury hospitalization as a negative control outcome in investigating the protective effect of influenza vaccination against influenza hospitalization and all-cause mortality in the elderly. They found that influenza vaccination appeared to be associated with a lower risk for both influenza hospitalization and all-cause mortality as well as trauma or injury hospitalization, indicating inadequate confounding adjustment. In our negative control outcome analyses, the aHR for trauma or injury hospitalization was close to the null for the three COVID-19 vaccines, suggesting that the negative association between COVID-19 vaccines and non-COVID-19 mortality was not likely biased by the pathways examined through the negative control outcome.
The associations that we found between COVID-19 vaccination and non-COVID-19 mortality are stronger than can plausibly be attributed to any real protective effect of vaccination. A more convincing explanation is selection bias as has been reported in studies of influenza vaccination and mortality. [18,19,40,41] Selection bias can arise as patients who anticipate that they are near death ''give up" on vaccinations as they are near death and they tend to become less willing and able to seek vaccinations and other preventive services. Although we have extensive data on diagnoses, Table 3 Characteristics of Ad26.COV2.S recipients and their comparators during the period from December 14, 2020 to June 30, 2021. demographics, and use of health services in the study population, this source of bias is not well measured, and we have not been able to adequately adjust for it. In the context of widespread suggestions on social media that COVID-19 vaccines are unsafe, it is reas-suring that we found no evidence of any association of COVID-19 vaccination with increased risk of death. We think our analyses would yield more convincing hazard ratio estimates if we could better adjust for selection bias. Future analyses using a modified self-controlled case series design might be able to mitigate the healthy vaccinee effect by controlling for unmeasured fixed risk factors through within-person comparisons. [42]. In addition to unmeasured confounding, this study had at least two additional limitations. First, causes of death were not available and were not included in the analyses. A temporal relationship between a COVID-19 diagnosis or a positive SARS-CoV-2 test and death was used as a proxy for defining COVID-19-related death. We could have missed COVID-19 related diagnoses and misclassified some non-COVID-19 deaths, especially among unvaccinated individuals because they were more likely to be infected with COVID-19. The potential differential misclassification of non-COVID-19 deaths may have overestimated the non-COVID-19 mortality rates among unvaccinated individuals, leading to lower hazard ratios for vaccinees. Further, without knowing causes of death, we could not estimate and compare the proportions of deaths due to various causes. Second, the VSD population is an insured population and the findings in the current study may not be generalizable to the general population.
Our study had several strengths. First, individual-level and community-level socioeconomic confounders were adjusted for in the survival analyses for estimating the association between  COVID-19 vaccination and non-COVID-19 mortality and all-cause mortality. In particular, we included inpatient and ED visits within 7 days prior to the index date (yes/no) and a frailty score in the propensity score models to control for healthy vaccinee effects. Second, we used a rigorous propensity score approach to adjust for the measured confounders. After applying stabilized weights to the cohorts, all measured confounders were well balanced between recipients of COVID-19 vaccines and their comparator groups. Third, the frequency matching of vaccinated individuals during a given week with comparators who had not been vaccinated yet aligned the start of the comparators' follow-up with that of vaccinated individuals. Because of the proper alignment of start of follow-up, the frequency matching helped to mitigate immortal time bias. [43][44][45] Fourth, the assignment of index dates for unvaccinated comparators that corresponded to the vaccination dates of their matched vaccinees, and the use of calendar time as the basic time scale in survival analyses ensured control for temporal factors. Finally, the study had a large, demographically diverse study population with up to 8 months of follow-up. We conclude that, while residual confounding bias remained after adjusting for several individual-level and community-level risk factors, no increased risk was found for non-COVID-19 mortality and all-cause mortality among recipients of three widely used COVID-19 vaccines in the US. The findings in this study of individuals 12 years and older support CDC's recommendation of COVID-19 vaccination for this age group. Future studies will include chil-dren<12 years of age. Disclaimer The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.   This study was approved by institutional review boards of all participating health care organization sites with a waiver of informed consent and was conducted consistent with federal law and CDC policy. §.

Data availability
Data will be made available on request.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.