There is an urgent need for the development of a safe and effective vaccine for COVID-19. As such, COVID-19 vaccine candidates have been developed at unprecedented speed, and there are currently a handful of vaccines in the final stages of clinical testing. It is of critical importance that the normal evaluation of safety is not reduced or compromised in any way because of high-speed clinical development.

UK trials of the Oxford and AstraZeneca vaccine have resumed after a brief pause, following safety concerns. A statement made by AstraZeneca on 12 September stated that “on 6th September, the standard review process triggered a voluntary pause to vaccination across all global trials to allow review of safety data by independent committees, and international regulators. The UK committee has concluded its investigations and recommended to the MHRA that trials in the UK are safe to resume.” A press release published on the AstraZeneca website on 2 October, 2020 confirmed that the trial had not yet been resumed in the USA [1]. AstraZeneca and the University of Oxford have not disclosed details of this case. More recently, on the 13 October, 2020, Johnson & Johnson reported that further dosing in the phase III ENSEMBLE COVID-19 vaccine trial had been paused, owing to an unexplained illness in a study participant [2]. In this editorial, we describe how serious adverse events reported in a vaccine clinical trial should be investigated by reviewers [regulators or members of the Data and Safety Monitoring Board (DSMB)] to assess whether a causal relationship exists.

1 Criteria for Assessment of Causality

The World Health Organization (WHO) issued updated guidelines for the assessment of adverse events following immunization (AEFI) in 2018 [3]. Whilst causality can be assessed at both the population and the individual level, this editorial focuses on the latter. The WHO guidance states that at the individual level it is usually not possible to establish a definite causal relationship between a particular AEFI and a particular vaccine on the basis of a single AEFI case report. It also emphasizes the initial requirement to have a valid diagnosis for the reported AEFI, which could be an unfavorable or unintended sign, an abnormal laboratory finding, a symptom, or a disease, and that, where possible, this should meet standard case definitions [3].

The Brighton Collaboration is an international voluntary organization to enhance immunization safety. In 2004, The Brighton Collaboration initiated a project to facilitate comparability of immunization safety data by developing standardized case definitions for AEFIs and guidelines for case determination, recording, and data presentation [4]. The Coalition for Epidemic Preparedness Innovation has partnered with the Brighton Collaboration, through the Task Force for Global Health, to harmonize the safety assessment of Coalition for Epidemic Preparedness Innovation-funded vaccines via its Safety Platform for Emergency vACcines (SPEAC) Project, launched in 2019 [5]. To date, 64 case definitions have been published for adverse events of special interest (AESI) following immunization, including guidelines for the collection, analysis, and presentation of data related to each outcome [6].

2 World Health Organization Algorithm

There are several models, algorithms, and tools (including software) available for causality assessment, each with its own merits and with varying sensitivity and specificity. After a thorough review of the existing methodologies for assessing causality in adverse drug reactions and AEFI, and after pilot testing of several approaches (including scoring scales, algorithms, and questionnaires) a Global Advisory Committee on Vaccine Safety working group in consultation with experts developed the approach outlined in the recent guidance [3]. Once the diagnosis has been validated, it is recommended that the reviewer works through a logical checklist, summarized below:

figure a

The boxes on the pathway shown in Fig. 1 below correspond to the four major sections in the checklist above. During the initial stages of the assessment when considering the eligibility, the reviewer may consider the available information to be sufficient for initiating the causality assessment process. However, after completing the checklist it may be discovered that the information is insufficient to arrive at a definite conclusion. At this stage of the review, the reviewer may decide to categorize the case as “unclassifiable” and specify the missing information that prevents the classification of the case [3].

Fig. 1
figure 1

Causality assessment algorithm [3]

Full size image

The final classification has been adapted from the “Definition and Application of Terms for Vaccine Pharmacovigilance” report of the CIOMS/WHO Working Group on Vaccine Pharmacovigilance [7]. The cause-specific definitions provide clarity on “A. Consistent causal association to immunization” and “C. Inconsistent causal association to immunization” (coincidental). The association is considered “B. indeterminate” when adequate information on the AEFI is available but it is not possible to assign it to either of the above categories. The WHO has developed an electronic software that can assist with the AEFI causality assessment process.

3 Further Recommendations

Further detail relevant to each of the major sections outlined in the checklist is provided below. The preliminary step involves an attempt to validate the diagnosis prior to assessing causality. As part of this process, it is suggested that all relevant information about the event is obtained: full clinical details, laboratory tests, and other investigations in chronological sequence, including any results obtained prior to the vaccination. The ultimate outcome of the event and any sequelae also need be confirmed; was it fatal, is it ongoing, did the patient recover (partially or fully), and when were these outcomes observed? All of this information will contribute to validation of the underlying diagnosis, which may change over time as additional information becomes available.

  1. (i)

    Is There Evidence for Any Other Causes?

    To decide whether there is evidence for any other cause, full information about the patients’ previous medical history should be obtained, including any illnesses diagnosed prior to and at the time of the event. Family history should also be sought, and enquiries made to ascertain whether the patient experienced any known risk factors for the specific adverse event of interest.

  2. (ii)

    Is There a Known Causal Association with the Vaccine or Vaccination?

    Enquiries to ascertain whether there is evidence to support a known causal association with the vaccine or vaccination should involve a literature search for all reports of the specific event of interest following vaccination. Where available, information on the outcomes in these patients should be ascertained, and how the particular serious adverse event affected the development and use of any associated vaccines established. It is helpful to further widen the search by conducting a literature review for similar events that have been reported following the administration of vaccines, with particular note of whether implicated vaccines shared the same vector, excipients, or adjuvants as the vaccine under study. In addition, further evidence can be collected through a detailed search of similar events in the study and the whole program, even if these events were considered to be unrelated previously, involving collection of all relevant case details to validate the diagnosis, including the clinical path of the condition, and a review of the patients’ past medical history. A database search for all other similar or related events using a broad search strategy should be conducted, ensuring the inclusion of all relevant event terms related to the relevant system organ class is adopted.

    It is recommended that assessment of biological plausibility should be considered following discussion with relevant experts and using knowledge of the underlying on- and off-target pharmacological effects. Additional enquiries to ascertain whether the vaccine may have a quality defect should be considered, i.e., was the occurrence of the event related to the batch, or any potential manufacturing issue degradation of the product during distribution and storage? The WHO guidance states that an immunization error describes an AEFI that is caused by inappropriate vaccine handling, prescribing, or administration and that therefore, by its nature, is preventable [3]. The types of reactions caused by immunization stress responses include, but are not limited to, acute stress responses, vasovagal reactions, and conversion disorders. Furthermore, it is important to confirm whether the event took place within a “plausible” time window of increased risk. This is applicable to all questions under (ii) of the checklist.

  3. (iii)

    Is There Strong Evidence Against a Causal Association?

    Is there a body of published evidence (systematic reviews, Global Advisory Committee on Vaccine Safety reviews, Cochrane reviews) against a causal association between the vaccine and the event?

  4. (iv)

    Other Qualifying Factors

    The guidance states that Sections (i)–(iii) outline the strong evidence for or against causality for most cases of AEFI. The checklist above includes some additional factors that support the above observations. If the AEFI is still unclassified, these qualifying factors provide reviewers with indications on causality.

    It would be the responsibility of the DSMB to evaluate the outputs of these enquiries and make recommendations based on the findings. A DSMB consists of a group of independent experts external to a study assessing the progress, safety data and, if needed, critical efficacy endpoints of a clinical study. Recommendations made by the DSMB can determine the fate of the study. The core objectives of a DSMB include the following:

    1. 1.

      Protection of the health of both current vaccinees enrolled in the study/program and future participants.

    2. 2.

      To ensure that the DSMB actions are both scientifically correct and proportionate to protect patients without compromising the study or the program.

    3. 3.

      To maintain the blinding of the study to the sponsors (not to compromise the study) and to ensure continuation of the integrity of the data.

    4. 4.

      To ensure that high scientific and ethical standards are maintained throughout the conduct of the study by the sponsor, investigators, and all other staff associated with the study.

      To maintain the integrity of the study, the sponsors, the study staff, and investigators remain blinded to the data; however, DSMB members are unblinded. Whilst this editorial provides an overview of best practice with respect to a causality assessment of serious adverse events in a vaccine study, it is acknowledged that regulators or members of the DSMB may be forced to act even in the absence of a definite causal relationship. At the end of every meeting, the DSMB would recommend one of the following courses of action:

      • To continue the study with no modification of the study protocol

      • To continue the study with a protocol modification(s), which may include:

        • Possible exclusion of certain groups of people from the vaccination program that may be at increased risk of developing the serious adverse event of interest

        • Additional investigations or screening criteria prior to including participants in the study

      • Request further information, either urgently or later with an agreed timeline

      • Co-opt an external expert(s) to advise on specific events

      • To suspend the study

      • To stop the study

    The DSMB may also elect to conduct more frequent reviews of the data. Following the review of data in the Oxford Study during the pause in the trial, the DSMB and UK regulators recommended the trial could restart. The DSMB will continue to be crucial in this study, to continually assess through review of the safety data whether the potential benefits from the vaccine continue to outweigh any associated risks.