Defining and reporting adverse events of special interest in comparative maternal vaccine studies: a systematic review

Introduction The GAIA (Global Alignment on Immunisation Safety Assessment in Pregnancy) consortium was established in 2014 with the aim of creating a standardised, globally coordinated approach to monitoring the safety of vaccines administered in pregnancy. The consortium developed twenty-six standardised definitions for classifying obstetric and infant adverse events. This systematic review sought to evaluate the current state of adverse event reporting in maternal vaccine trials following the publication of the case definitions by GAIA, and the extent to which these case definitions have been adopted in maternal vaccine safety research. Methods A comprehensive search of published literature was undertaken to identify maternal vaccine research studies. PubMed, EMBASE, Web of Science, and Cochrane were searched using a combination of MeSH terms and keyword searches to identify observational or interventional studies that examined vaccine safety in pregnant women with a comparator group. A two-reviewer screening process was undertaken, and a narrative synthesis of the results presented. Results 14,737 titles were identified from database searches, 435 titles were selected as potentially relevant, 256 were excluded, the remaining 116 papers were included. Influenza vaccine was the most studied (25.0%), followed by TDaP (20.7%) and SARS-CoV-2 (12.9%). Ninety-one studies (78.4%) were conducted in high-income settings. Forty-eight (41.4%) utilised electronic health-records. The majority focused on reporting adverse events of special interest (AESI) in pregnancy (65.0%) alone or in addition to reactogenicity (27.6%). The most frequently reported AESI were preterm birth, small for gestational age and hypertensive disorders. Fewer than 10 studies reported use of GAIA definitions. Gestational age assessment was poorly described; of 39 studies reporting stillbirths 30.8% provided no description of the gestational age threshold. Conclusions Low-income settings remain under-represented in comparative maternal vaccine safety research. There has been poor uptake of GAIA case definitions. A lack of harmonisation and standardisation persists limiting comparability of the generated safety data.


Introduction:
The GAIA (Global Alignment on Immunisation Safety Assessment in Pregnancy) consortium was established in 2014 with the aim of creating a standardised, globally coordinated approach to monitoring the safety of vaccines administered in pregnancy.The consortium developed twenty-six standardised definitions for classifying obstetric and infant adverse events.This systematic review sought to evaluate the current state of adverse event reporting in maternal vaccine trials following the publication of the case definitions by GAIA, and the extent to which these case definitions have been adopted in maternal vaccine safety research.Methods: A comprehensive search of published literature was undertaken to identify maternal vaccine research studies.PubMed, EMBASE, Web of Science, and Cochrane were searched using a combination of MeSH terms and keyword searches to identify observational or interventional studies that examined vaccine safety in pregnant women with a comparator group.A two-reviewer screening process was undertaken, and a narrative synthesis of the results presented.Results: 14,737 titles were identified from database searches, 435 titles were selected as potentially relevant, 256 were excluded, the remaining 116 papers were included.Influenza vaccine was the most studied (25.0%), followed by TDaP (20.7%) and SARS-CoV-2 (12.9%).Ninety-one studies (78.4%) were conducted in high-income settings.Forty-eight (41.4%) utilised electronic health-records.The majority focused on reporting adverse events of special interest (AESI) in pregnancy (65.0%) alone or in addition to reactogenicity (27.6%).The most frequently reported AESI were preterm birth, small for gestational age and hypertensive disorders.Fewer than 10 studies reported use of GAIA definitions.Gestational age assessment was poorly described; of 39 studies reporting stillbirths 30.8% provided no description of the gestational age threshold.Conclusions: Low-income settings remain under-represented in comparative maternal vaccine safety research.There has been poor uptake of GAIA case definitions.A lack of harmonisation and standardisation persists limiting comparability of the generated safety data.

Introduction
Neonatal mortality continues to pose a significant global health challenge, accounting for almost half of all under-5 deaths worldwide [1].While significant progress has been made in reducing child mortality, neonatal mortality remains a persistent concern, particularly in low-and middle-income countries (LMICs).Maternal vaccination has emerged as a critical intervention with the potential to substantially impact neonatal health outcomes.New vaccines are under development to address key pathogens impacting infant mortality [2].The potential of this approach has been evidenced by the maternal-neonatal tetanus program in which the well-established tetanus vaccine is one of the key interventions [3].First implemented in 1989, by 2020, all but 12 targeted countries had reached elimination status [4].Vaccinations during pregnancy also help safeguard the mother from vaccine-preventable illnesses such as influenza and SARS CoV-2, reducing the risk of severe complications of diseases and associated adverse outcomes [5].
According to the World Health Organization (WHO), Immunisation currently prevents 3.5-5 million deaths every year from diseases like diphtheria, tetanus, pertussis, influenza and measles [6].However, despite the available evidence of effectiveness and established reassuring safety profiles of vaccines, vaccine hesitancy has emerged as a concerning global phenomenon [7].The rise in vaccine hesitancy, which has been demonstrated in pregnancy, poses a grave threat to public health, jeopardising the significant progress achieved through immunisation programs.Demonstrating a commitment to vigilance, transparency, and evidence-based decision-making, vaccine safety research builds public trust in vaccines.Such systems provide reassurance that the safety of individuals and communities is a top priority throughout the entire vaccine life cycle, fostering confidence in the benefits of immunisation and ultimately contributing to higher vaccine acceptance rates.
The Brighton Collaboration, an international network of experts established in 2000, has developed standardised case definitions for adverse events following immunisation (AEFI) in order to provide a common language and framework for identifying, classifying, and reporting vaccine-related adverse events.Their goal is to improve the collection and analysis of vaccine safety data through the use of consistent terminology and criteria to enable comparison across different studies, surveillance systems and settings.
The GAIA (Global Alignment on Immunisation safety Assessment in pregnancy) consortium was formed as part of the Brighton Collaboration, to address the specific need for standardised methods for assessing the safety of research vaccines during pregnancy.A systematic review to assess variability in AEFI reporting was conducted as a preliminary step in development of these standardised definitions.This review identified variability in how the presence of AEFIs were determined, how AEFI definitions were applied, and in the ways that AEFIs were reported.Definitions for AEFIs differed in terms of level of detail, boundaries and cut-offs, severity strata and standards used [8].The GAIA group sought to address these issues through development of standardised maternal, fetal and infant event definitions for the classification of adverse events of special interest (AESI) [9].These case definitions were developed with the aim of achieving sufficient applicability to be of use in monitoring immunisation safety in pregnancy globally [10].The selection of these outcomes was prioritised based on recommendations from global experts convened by the World Health Organization in 2014 [10].The first ten definitions were published in 2016, these included stillbirth, neonatal death, maternal death, congenital anomalies, non-reassuring fetal status, hypertensive disorders in pregnancy, pathways to preterm birth, postpartum haemorrhage, preterm birth and neonatal infections.These were followed by the next twelve definitions in 2017 -congenital microcephaly, fetal growth restriction, antenatal bleeding, dysfunctional labor, gestational diabetes, spontaneous abortion, ectopic pregnancy, neonatal encephalopathy, failure to thrive, low birthweight, respiratory distress and small for gestational age.A further four which include chorioamnionitis, postpartum endometritis, neonatal seizures and neurodevelopmental delay were published in 2019 [11].The definitions categorise the outcomes into levels of diagnostic certainty (1-3), with greatest specificity at the highest level (level 1) and increasing sensitivity as you progress through the levels, whilst still maintaining an acceptable specificity.They were developed in this way to accommodate the resources and diagnostic capabilities available in different locations.
This updated review aimed to assess the status of maternal vaccine AEFI reporting in comparative clinical vaccine trials following publication of the standardised case definitions by GAIA.We assessed the extent to which these guidelines and case definitions have been adopted in maternal vaccine safety research since their publication seven years ago.

Methods
The study objectives were to describe the study characteristics, types of vaccines researched in pregnancy, the frequency of reported adverse event outcomes, classification, or provision of definitions for the adverse event outcome and use of Brighton Collaboration GAIA definitions.We also aimed to describe the consistency observed in AEFI reporting, variability in definitions utilised and the adverse event data collection methodologies.The study was registered on the international prospective register of systematic reviews (PROSPERO) CRD42021253680.

Eligibility criteria
Studies, whether interventional or observational, that involved administration of any vaccine(s) to pregnant women of any age were included in the review.Studies that did not include pregnant women, either as the main participants or as an at-risk group were excluded.Studies making any relevant comparison of vaccines against a control, such as placebo, alterative vaccine, unexposed or untreated group, were included, studies that did not include a comparator were excluded.Acceptable outcome measures included intervention efficacy, effectiveness, or safety.Studies that did not evaluate vaccine safety as a primary outcome were included if maternal, or neonatal safety or adverse event data were presented.The study setting had no impact on inclusion.Studies conducted in any language other than English were excluded as were unpublished studies.Studies were limited to those published between the date of the previous review by Fulton et al [8] (2014) and the date of the search.

Search strategy
A systematic search of published literature potentially containing data on maternal and neonatal adverse events following maternal immunisation was conducted.All published comparative maternal immunisation studies (randomised controlled trials and observational studies), identified via searches of PubMed, EMBASE, Web of Science, and the Cochrane Database using a combination of medical subject headings (MeSH) terms and keyword searches were included.The search strategy used for this review was derived from prior work by Fulton et al [8].Supplementary Tables 1, 2, 3, 4 outlines the search strategies in the four databases searched.Results were limited to English language publications between 2014 and December 31st, 2022.

Selection & data extraction
A two-reviewer system was adopted for the entire review process.Results of the database searches were imported into Mendeley Desktop and automated deduplication followed by manual deduplication was undertaken.Two reviewers (HGD + KK, LH or ET) screened all titles and abstracts for eligibility.Articles were excluded if their titles and abstracts were clearly unrelated to the criteria of this review.Full texts of all eligible studies were retrieved and reviewed by two reviewers (HGD, KK or ET), studies for which full texts were inaccessible were discarded.The rationale for study exclusion was recorded as part of the screening process.Consensus or recourse to a third review reviewer (KLD) occurred in the case of uncertainty during the screening process with regards to inclusion/exclusion.An adapted PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow-chart of study selection was completed (Fig. 1 & supplementary Fig. 1) [12].
The objective of this review was to determine the variance in AEFI definitions across all comparative maternal immunisation studies within the prespecified dates, irrespective of study design, rigor, outcome, or potential bias.A methodological study quality assessment (Grading of Recommendations Assessment, Development and Evaluation (GRADE) analysis) was, therefore, not needed.Studies included were not assessed for, nor ranked based on limitations in design or possible bias.A narrative synthesis of the evidence on AEFI reporting in maternal vaccine studies was undertaken.All data from studies meeting the inclusion criteria were extracted into an Excel workbook for analysis.Advanced data analysis, when required, was conducted in Stata version 18.0.

Study selection & characteristics
14,737 titles were identified from electronic searches, 435 titles were selected as potentially relevant, 432 were retrieved and assessed for eligibility.Following assessment, 256 papers were excluded, and the remaining 176 papers were included in the review (Fig. 1).116 of these were original research papers (Table 1) and 60 review papers (Supplementary Table 5).
Retrospective cohort studies made up the majority in terms of study design (50.9%), twenty-four reported on randomised controlled trials (20.7%) and 23 (19.8%) were prospective cohort studies.Forty-eight studies (41.4 %) utilised electronic health records to identify AEFI.Most of the included studies focused on adverse events of special interest in pregnancy alone (65.5%), 32 (27.6%) assessed vaccine reactogenicity additionally and 6.9% focused solely on reactogenicity.Table two details these and other key characteristics of each study.
Most studies were conducted in high-income settings (78.4%).Of note, only 12 (10.3%)studies were exclusively conducted in lower-or lower-middle-income countries.Studies conducted in high-income settings tended to be larger than those in lower-income settings with a mean number of participants ten times higher than in low to uppermiddle-income settings (11,162 v 996).No EMR or registry-based studies conducted in LMIC settings were identified.pregnancy (19,430 events) and chorioamnionitis (18,273) were the most frequently reported events in maternal subjects.Small for gestational age (30,929), preterm birth (28,828) and congenital anomalies (13,466) were the most frequent neonatal outcomes.There were many reported adverse events that have not been systematically defined, these include neonatal jaundice (17,166), macrosomia (2020), autism spectrum disorders and attention deficit hyperactivity disorders (1609) (Supplementary Table 6).

Utilisation of brighton collaboration definitions
Eight of the included studies cited or used the Brighton Collaboration GAIA definitions in their publications (Table 4).Two studies cited the GAIA definitions in their discussion sections as a way of improving the quality of their respective studies but did not use the definitions [13,14].Three studies [15][16][17] used the Brighton Collaboration's 2015 publication [10] of key terms for the assessment of safety of vaccines in pregnancy to guide the outcomes they investigated and reported in their studies but did not use the published definitions; two of these were EHRbased and used ICD codes to identify cases [15,16], one was a randomised controlled trial [17].Three studies used the definition or a modified version of it.One of these used the GAIA stillbirth definition to differentiate between antepartum and intrapartum stillbirth but used other criteria for gestational age assessment [18].A further study reported on several GAIA outcomes but modified some of the criteria, for example, including multi-organ complications (including small for gestational age) as one of the criteria for diagnosing pre-eclampsia [19].Only one published study provided a level of certainty associated with the reported outcomes [20].

Thresholds and cut-offs
Criteria for many of the defined events depend on temporal or physiological thresholds for determining the presence of an event.These include, for example, the size of measured induration around a vaccine site reaction or the gestational age at which a fetal loss is classified as a stillbirth as opposed to a spontaneous abortion.Critically, several of the GAIA outcomes depend on accurate assessment of the gestational age of the pregnancy.The method of gestational age assessment was infrequently reported, even in randomised controlled trials where 55% did not provide the assessment methodology in their publication (or accompanying protocol where available).
Of the 39 studies that reported stillbirths, 12 different thresholds were published, these varied in terms of gestational age and/or weight thresholds, whilst 12 studies (30.8%) did not provide a definition at all (Supplementary Table 7).Similarly for preterm birth, 10 publications (14.7 %) did not provide a definition of the gestational age below which the birth was considered preterm (Supplementary Table 8).

Serious adverse event reporting
Twenty-four publications reported on 22 different RCTs (Supplementary Table 9).Nineteen of these reported on serious adverse events in their published manuscripts.Eight studies (42.1%) provided a definition of an SAE in the manuscript or accompanying published protocol or provided a reference to the different standards they used for  categorising SAEs.There were variations in the definition used; two defined an SAE as an event that was considered life threatening, prolonged hospitalisation, resulted in persistent or significant disability, or resulted in death [21,22].Four studies maintained that definition with the additional inclusion of congenital anomalies or birth defects [17,[23][24][25].Additionally, two studies included all outcomes of special interest in pregnancy [26] as serious adverse events or a selection (spontaneous abortions, fetal death, stillbirth) [27] of adverse outcomes of special interest as SAEs.

Discussion
This review has demonstrated that almost eight in ten comparative vaccine safety studies published between 2014 and the end of 2022 were undertaken in high-income settings.Where studies were set in lower-income settings, they tended to be smaller, recruiting fewer participants and not utilising large health registries or electronic sources.There are several reasons why vaccine safety concerns may differ by setting; safety profiles may differ based on factors such as nutritional status, presence of coexisting infections such as HIV, genetic factors, and exposure patterns.By conducting maternal vaccine safety studies in lowresource settings, our understanding of the safety profile of vaccines in diverse populations can be enhanced, addressing specific safety concerns relevant to these settings, and ensuring that vaccine recommendations are inclusive of all populations and evidence-based.These benefits can only be realised by improving the equitable distribution of vaccine research globally and investing resources in conducting larger vaccine safety studies in LMICs.
Over half of the studies that reported comparative vaccine safety in pregnant women identified in this review utilised retrospective study designs.The GAIA definitions were designed to be used prospectively in clinical trial settings, and may not be applicable in studies that utilise retrospective case identification.A 2022 review demonstrated that large numbers of cases identified retrospectively, and which utilised routine care data were unclassifiable using the GAIA definitions [9].Safety data in pregnant women relies heavily on post-marketing pharmacovigilance and retrospective datasets such as electronic health-records, and billing codes on administrative data are commonly used in these study designs.Case definitions that can be applied to retrospective datasets and from regular clinical data are also needed.The GAIA definitions might need to be adapted for use in this context.
Significant work to evaluate the GAIA definitions has been undertaken, however, a recent review identified that half had yet to undergo formal evaluation in published studies [9].These include some of the most frequently reported maternal (chorioamnionitis and post-partum haemorrhage), fetal (spontaneous abortions) and neonatal (congenital anomalies) AESIs identified in our systematic review.Work to complete validation of the definitions is planned and these commonly reported outcomes should be prioritised in future validation studies.Several recent publications have added to the weight of evidence in this area.Two studies from Democratic Republic of Congo have assessed the definitions in retrospective [28] and prospective [29] datasets and will help progress the validation process.
Our review also identified some commonly reported AESI that have not been systematically defined by the Brighton Collaboration, these include neonatal jaundice, macrosomia, autism spectrum disorder and attention deficit hyperactivity disorder.The comprehensive list of all reported AESI from maternal vaccine studies generated by this review could support identification of other outcomes beyond the 26 already published that require attention.
In agreement with the findings of the 2015 systematic review; there  were substantial variations in temporal and physiological thresholds for reporting of adverse event outcomes, particularly around gestational age.Given that gestational age assessment is critical in maternal vaccine studies, not only for assessment of outcomes such as stillbirth, preterm birth and small for gestational age, but also for the actual timing of vaccine delivery, significant improvements could be made in reporting of the methodologies used in maternal vaccine studies.Our review identified that 14.7% of studies did not provide this information on the gestational age threshold for preterm birth and 55% of RCTs did not provide information about how the pregancy was dated in their published manuscripts.This information is required so that a transparent assessment of the level of confidence associated with reported maternal outcomes such as hypertensive disorders, fetal outcomes such as fetal growth restriction and spontaneous abortion and neonatal outcomes such as small for gestational age can be made.
A 2012 systematic review of safety reporting in developing country vaccine clinical trials identified 50 RCTs between 1989 and 2011.AEFI definitions were used in 35 of 50 vaccine trials.Standardised Brighton Collaboration definitions were used in two trials.Logistic regression revealed a positive association between use of a fever case definition and the reporting rate for fever as an AEFI (p = 0.027) suggesting that use of a definition may increase the likelihood that the outcome was reported [30].
Seventy-nine studies (68.1 %) were published after the first set of GAIA definitions in 2016, but despite this, we have demonstrated poor uptake of the definitions even in the prospective RCTs for which they were developed.Similarly poor uptake of Brighton Collaboration definitions was identified in a 2019 study in low-and middle-income settings.The authors looked at all Brighton Collaboration definitions, not just those specifically relating to pregnancy and concluded that the Brighton Collaboration case definitions had not been broadly used or assessed for performance in low-and middle-income countries (LMICs).They recommended involving more LMIC scientists in the case definition working-groups, disseminating free webinars, publicly available recordings, and case definition documents.They also suggested formally assessing barriers for use in LMICs; and developing tools to support implementation [31].Our review suggests that the problem is not limited to LMICs, and such activities should cover a wider range of settings.
Completion of the planned evaluation studies and undertaking critical revisions that have been identified for poorly performing definitions, such as stillbirth, could also improve confidence in the definitions and support more widespread adoption [32].Given the importance of consistency of safety reporting, consideration should also be given to making use of the definitions by investigators a requirement.The CONSORT (Consolidated Standards of Reporting Trials) statement; a structured framework for researchers to report their study methods and findings transparently and accurately, helps improve the quality and reliability of published research [33].Whilst use of the framework isn't mandatory, many medical journals and publishers strongly encourage or require authors to follow the CONSORT guidelines when submitting RCTs for publication.Moreover, many research funding agencies and ethics committees also recommend or require compliance with the CONSORT guidelines when researchers apply for grants or ethical approvals.Given the importance of robust vaccine safety monitoring, and the limited uptake of the definitions, a similar approach should be considered for standardised reporting of adverse events of special interest in maternal vaccine trials once the evaluation and revision process has been completed.
Our search was undertaken in four major literature databases, but we did not hand-search individual journals or grey literature which may have led to us missing a small number of relevant papers.In addition, it was not feasible to assess vaccine studies that did not include a comparator group, studies undertaken using passive surveillance registries, such as VAERS, were therefore excluded.Studies were limited to those published in English and so relevant studies from LMIC settings written in other languages may have been missed.We think that this is likely to have a limited impact on the overall number of comparative safety studies identified from LMICs.Fulton et al did not limit their database search to English and only identified 5/5488 (0.09 %) potentially relevant non-English language studies [8].It was also beyond the scope of our study to contact the authors of each study to request additional, unpublished information regarding AEFI definitions, we did however, review any protocols accompanied alongside the main manuscript.

Conclusions
Research into vaccine safety continues to be undertaken predominantly in high-income regions, while low-income countries receive insufficient attention, both in terms of the quantity of studies conducted and the number of participants involved.Vaccines against diseases such as Ebola and malaria, that may be of particular benefit to pregnant women, are being developed for, and introduced in, high-burden LMIC settings.To ensure that the introduction of these vaccines is supported by adequate safety data, it is essential to prioritise the establishment of capacity and infrastructure for conducting rigorous studies in these settings.
There remains a persistent issue of inconsistency, variability, and a lack of transparency in reporting adverse events in maternal vaccine studies.Furthermore, the adoption of GAIA (Global Alignment of Immunization Safety Assessment in pregnancy) AESI case definitions, since their publication, has been disappointingly low, with only one study providing a level of confidence associated with their reported outcomes.However, it is important to note that even though the majority of studies were published after the definitions, a large number of the studies included in this review began recruitment before their publication, and it may take some time for widespread adoption to occur.
To address this, there is a need to prioritise the evaluation of the thirteen GAIA definitions that have not yet been tested in published studies.This step will ensure that these definitions have undergone thorough assessment, instilling confidence in researchers to use them.Additionally, the planned review and revision of the case definitions may contribute to greater adoption.Finally, consideration should be given to ways in which the definitions and associated materials can be disseminated more widely within the research community.
Developing Countries Clinical Trials Partnership.Clare Cutland and Flor Muñoz were involved in leading the GAIA definition development.Hannah Davies is involved in the GAIA definition revision process.

Table 3
outlines the frequency that AESIs defined by the GAIA-group were reported in the included studies.Hypertensive disorders of Fig. 1.PRISMA Flow Diagram.H.G.Davies et al.

Table 1
Description of all included studies (excluding review papers).
21uble dose and two single doses of inactivated influenza vaccine compared with a single dose of inactivated influenza vaccine in pregnant women living with HIV to each of the three vaccine strains, and relative safety of the three dosing schedules.21Munozetal.[21]2020 North America Influenza HI Injection site and systemic reactions in the pregnant women for 7 days following receipt of the study vaccine; vaccine-associated maternal AEs and maternal and infant SAEs for the duration of study participation; and pregnancy outcomes, including maternal and neonatal complications during pregnancy and at time of delivery.(continued on next page) H.G. Davies et al. (continued on next page) H.G. Davies et al.

Table 2
Summary characteristics of included studies.

Table 4
Publications that cited or utilised GAIA Definitions.

Table 3
Numbers of studies reporting GAIA adverse outcomes and frequency of the outcomes reported in vaccine recipients in included studies.