Methods and challenges in measuring the impact of national pneumococcal and rotavirus vaccine introduction on morbidity and mortality in Malawi

Highlights • Evaluation of vaccine impact and effectiveness is critically important but methodologically challenging.• Challenges include achieving unbiased ascertainment of vaccine status, and non-vaccine-attributable decline in morbidity and mortality.• Use of multiple sites together with diverse evaluation methods provides more robust evaluation.

but potentially significant contribution of vaccines to mortality prevention. Careful evaluation of vaccine effectiveness and impact in such settings is critical to sustaining support for vaccine programmes. Our evaluation platform covers a large population with a high prevalence of HIV and malnutrition and its findings will be relevant to other settings in sub-Saharan Africa.
© 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

Background
Key to reducing child mortality in developing countries has been the recent introduction of pneumococcal conjugate and rotavirus vaccines [1]. Their incorporation into routine infant schedules requires robust evidence from randomised efficacy trials. Demonstrating subsequent effectiveness and impact is challenging but essential for optimal implementation of these vaccines and sustained investment [2].
Pneumococcus is a leading cause of pneumonia, meningitis and bacteraemia [3]. In clinical trials pneumococcal conjugate vaccines (PCV) were extremely efficacious at preventing serious vaccine serotype disease [4][5][6]. In low disease burden high income countries, PCV have led to major public health benefits through direct effects and through herd immunity [7]. However, given differing epidemiology of pneumococcal transmission and exposure and vaccine scheduling in resource-poor settings and higher prevalence of malnutrition and HIV, vaccine effectiveness may differ. Furthermore, in several populations the overall benefit of PCV has been offset by an increase in non-vaccine serotype disease and doubts remain over the efficacy of the vaccine against some serotypes [8,9].
Rotavirus is the principal cause of severe diarrhoea globally and a major contributor to diarrhoeal deaths in the developing world [10]. Trials of oral rotavirus vaccines in high and middle income countries demonstrated high efficacy (>85%), with evidence of substantial public health impact demonstrated by post-introduction surveillance studies [11]. In low income countries these vaccines show lower efficacy [12,13]. A randomised trial of rotavirus vaccine conducted in Malawi found 49% efficacy [12], lower than in other studies [14,15]. Nonetheless, given the much higher disease burden and mortality, rotavirus vaccination has the potential to contribute to greater absolute improvements in child survival, particularly if indirect effects are apparent.
PCV and rotavirus vaccine were recommended by the World Health Organisation (WHO) in 2007 and 2009 respectively and subsequently became eligible for Gavi Alliance support for low income countries [16,17]. Malawi is among the poorest and most densely populated countries in Africa with a high burden of undernutrition, HIV, malaria, pneumonia and diarrhoea [18]. Increasing HIV prevalence among women age 15-49 of 13.3% in 2004 [19] to 22.7% in 2010 [20] led Malawi to adopt Option B+ (whereby all HIV-positive pregnant or breastfeeding women commence lifelong full antiretroviral therapy regardless of clinical or immunological stage) in 2011 [21,22]. Under 5 mortality is decreasing (79 per thousand live births in 2010) possibly due to increased birth spacing, antiretroviral roll-out, improved nutrition, sanitation and case management, placing Malawi on track to meet the fourth Millennium Development Goal of reducing child mortality [20]. With GAVI support, 13-valent pneumococcal conjugate vaccine (PCV13) was introduced into the national immunisation programme in November 2011 as a three-dose schedule (6, 10 and 14 weeks) with a catch-up dose in the first year of introduction for all infants <12 months old. Monovalent rotavirus vaccine (RV1) was introduced in October 2012 on a WHO recommended two-dose schedule (6 and 10 weeks).
Surveillance for invasive pneumococcal disease (IPD) or rotavirus acute gastroenteritis exists elsewhere in Africa, including the Gambia, South Africa, Ghana and Kenya amongst others [23,24].
Malawi is notable for being the only country in which PCV13 was introduced to a pneumococcal vaccine naïve population without a catch-up campaign after the first year of life, and for investigating the impact of both PCV and RV1 on infant mortality. We describe the methods of a nationwide surveillance platform that has been established to evaluate pneumococcal and rotavirus vaccine impact and effectiveness and consider the platform's strengths and weaknesses.

Study design
In collaboration with the Ministerial National Immunisation Technical Advisory Group (NITAG) [25], a national vaccine evaluation programme for vaccine-preventable endpoints has been established in Malawi to determine the population impact of new vaccines on mortality and morbidity. Several interlinked epidemiological methods have been used (Table 1) [26][27][28]. 5. Population based cohort study to measure <5 years and postneonatal infant all-cause and disease-specific mortality by vaccine status [29]. 6. Cost-effectiveness studies from a societal, health care provider and household perspective collecting comprehensive itemised costing of actual costs incurred [30].
Case definitions, screening criteria and investigations are shown in Tables 2 and 3. Universally issued subject-held medical records (health passports) which contain the vaccination records are interrogated for vaccination status.

Sites and populations under surveillance
The northern, central and southern regions of Malawi all participate in surveillance; the northern (Karonga) and central (Mchinji) sites are rural locations. The southern site (Blantyre) is a large city with high-density urban populations and a peri-urban rural penumbra, so surveillance is representative of the entire population and demography of Malawi (Fig. 1). The mortality cohort study is located in the northern and central regions, while the morbidity Table 1 Summary of study details.

Study Design
Primary endpoint (see also

13-valent vaccineserotype invasive pneumococcal disease
A suspected episode of septicaemia or meningitis with confirmed aetiology from CSF or blood by isolation of pathogen or detection by PCR based methods.

WHO defined radiological pneumonia
Among children <5 years presenting to a study site (either as outpatient or admitted inpatient), an episode of IMCI-defined (see Table 1) severe pneumonia or pneumonia with very severe disease that also has WHO defined endpoint consolidation as determined by concordant X-ray reading among a panel of 3 especially trained readers.

Severe acute rotavirus gastroenteritis
New onset of gastroenteritis with a modified Vesikari score ≥11 and with laboratory confirmation of rotavirus infection by enzyme linked immunosorbent assay.

All-cause post-neonatal infant mortality
A reported death of a child aged 4 to 52 weeks from the defined birth cohort capture population.
case control studies and cost-effectiveness studies are located in the northern and southern regions.  [31], health facility based pneumonia and diarrhoea surveillance in children operating since 2008 [31]. Births and deaths are reported monthly by a system of village key informants with verbal autopsies performed on all deaths. Health passport vaccine records are reviewed at an annual census. Children under surveillance attend Chilumba Rural Hospital for medical care. Those with Integrated Management of Childhood Illness (IMCI) defined severe pneumonia or gastroenteritis undergo chest X-ray or stool collection for rotavirus testing, blood cultures are not performed. Karonga contributes to studies 1, 2 (but not IPD), 4 and 5 listed above. The study site provides robust data on historical rates of disease and mortality prior to vaccine introduction.
Central Malawi: Mchinji district population was 456,516 persons in the 2008 census, with an estimated live birth rate of 57 per 1000 population [32,33]. We conducted a baseline census of Mchinji district in March 2012. Ongoing prospective surveillance of over 2000 villages in the district is conducted for births and deaths or migration of children under 5 and women of child-bearing age. Village informants report vital events monthly, supported by enumerators, supervised by monitoring and evaluation officers (Fig. 2). Live births are followed to 1 year of age or death or emigration from the district. All deaths reported by informants are verified and cause of death determined by verbal autopsy. Vaccine status is obtained from the health passport at household visits at 4 and 12 month of age or following death. Mchinji contributes to the population based cohort study listed above (study 5) and the primary endpoint of survival to 1 year. The site is coordinated by the Mai Mwana Project, with support from University College London.
Southern Malawi: The Malawi-Liverpool-Wellcome Trust Clinical Research Programme (MLW) in Blantyre has surveillance for IPD spanning 13 years [3,22], and has extensively characterised rotavirus molecular epidemiology in the pre-vaccine era [34,35]. Blantyre district numbered 1,001,984 persons at the 2008 census [32]. Queen Elizabeth Central Hospital (QECH) provides free health services and is the principal referral centre [36]. Active case finding at QECH paediatric emergency department and inpatient wards enrols children with presumed sepsis, meningitis, acute gastroenteritis and severe respiratory illness, in whom blood culture, lumbar puncture, stool collection, chest radiography or nasopharyngeal swabs are performed per protocol. Blood culture is not routinely performed in pneumonia unless a clinical suspicion of sepsis exists. Anthropometry, vaccination status, and HIV status are recorded [21]. This site is the platform for case-control studies of PCV13 effectiveness against vaccine type IPD and against radiological pneumonia and of RV1 against severe rotavirus gastroenteritis. The Blantyre site contributes to studies 1, 2, 3, and 4 listed above.

Sample size estimates
Vaccine effectiveness case-control studies: Assuming 60% vaccination coverage and control:case ratio of 4, the studies have 80% power to detect PCV13 effectiveness of ≥60% against vaccine type IPD with 47 vaccine age-eligible cases, ≥30% effectiveness against radiologically confirmed pneumonia with 311 cases, and effectiveness of ≥40% RV1 effectiveness against severe acute rotavirus gastroenteritis with 82 cases. Among adults, 311 vaccine type IPD cases living with vaccine eligible children are needed to detect ≥30% indirect effectiveness.
Mortality cohort study: Assuming 70% vaccination coverage and accounting for a 12% loss to follow-up before 1 year of age, we aim to recruit 38,213 live births over 2 years. This cohort provides 80% power at 5% one-tailed significance to detect vaccine associated improvement in cause-specific post-neonatal infant mortality of ≥36% if baseline mortality is 15 per 1000 live births.
Cost-effectiveness: A sample size of 88 for each clinical endpoint (confirmed rotavirus gastroenteritis, radiologically confirmed pneumonia) provides a precise estimator of the mean cost of illness episode to within a margin ±10% [37].

Implementation
Initial programme design involved close collaboration with the Ministry of Health, with specific inputs from the following offices: Expanded Programme for Immunisation, Acute Respiratory Infection branch, IMCI branch and Epidemiology branch. Site visits were conducted to assess research capacity. Regular meetings were held with the District Health Offices and Traditional Authorities. Study oversight is provided by the national coordinator based at MLW Blantyre, with site coordinators at each location. Regular site visits occur.

Capacity building and health system integration
All studies have dedicated employed staff. Clinical staff were trained in IMCI clinical management, Good Clinical Practice and formally qualified as HIV testers and counsellors. Training in paediatric radiological pneumonia standards was done against a WHO defined panel of standard films [38]. WHO verbal autopsy training was undertaken in Mchinji and Karonga. Sensitisation on childhood immunisation of community leaders and health care staff was undertaken. Review and feedback on quality of recording in vaccination centre registers was also conducted. Data are shared with the NITAG in which senior study staff are members, and Ministerial NITAG members are on the platform's steering committee.

Physical resources
Field: Coordination of activities across distant sites with poor accessibility demanded a robust motorbike and bicycle based transport infrastructure and regular site visits by supervisors to local field teams for scheduled data monitoring visits.
Laboratory: MLW has an external quality assured microbiology laboratory. Stool specimens received in Karonga are transported to MLW for further processing. Rotavirus status is determined by enzyme linked immunosorbent assay (Rotaclone TM , Meridian Bioscience, Cincinnati). Rotavirus genotyping is done using hemi-nested multiplex RT-PCR [39]. Pneumococcal quantification is performed using lytA PCR, while serotyping of carriage specimens from Karonga and Blantyre and of invasive isolates from Blantyre is performed using triplex PCR and latex agglutination [40]. HIV status of all children participating in studies 1, 2, 3, 4 and 6 is determined in accordance with national guidelines [21].
Radiology: Chest X-rays from Blantyre and Karonga are stored digitally according to a standardised protocol. Digital files are reviewed at each site and then centrally by a trained panel.

Data management
Data management has ensured harmonised linkage among the sites, whilst integrating with existing legacy systems [41]. Data are captured on paper forms and entered to a relational Structured Query Language database, with embedded data checks. In Mchinji paper forms are reviewed, scanned and entered. Timeliness is critical since subsequent contacts with subjects are automatically triggered by the database. Verbal autopsy data are captured using purpose built Visual Basic (Microsoft) platform. Similar systems occur in Karonga. Data cleaning, merging, logging and archiving is undertaken weekly by a dedicated team, whose performance is monitored. Subject recruitment by endpoint is reported at each site and is reviewed regularly by the study steering committee.

Quality assurance activities
For valid ascertainment of vaccine status, the health passport is reviewed upon each contact with participants. Passport images are scanned and digitally archived. In Karonga, digital capture occurs for all cases and a random selection of children in the DSS. In Mchinji, cluster sampling by catchment area is undertaken and a representative sample is imaged and archived. Vaccination records are checked against the local health centre vaccination register. Parental verbal report is also obtained for all children. Discrepancies are resolved on the basis of rules defined a priori in the study protocol. In case of missing passports, particularly among deceased infants, vaccination registers are cross checked. In Mchinji, interviews are conducted quarterly with a sub-set of mothers following vaccination visits to confirm whether the child received the number of vaccine doses that are recorded in the health passport.

Programme performance indicators
Recruitment and protocol compliance are reviewed weekly in Blantyre, and monthly in Karonga and Mchinji. Observed event rates are compared to long term expected rates and possible under-ascertainment is investigated by detailed review. Contiguous periods of suspected under-reporting will trigger the central database to issue a warning and restorative field activities undertaken to collect missing data.

External evaluation
Initial study protocols were reviewed by an external scientific panel on behalf of the Wellcome Trust. Subsequent formal regular external review of protocol compliance is provided by visiting epidemiologists (separate from study co-investigators) from the Centers for Disease Control and Prevention, Atlanta.

Ethics
Community approvals from Traditional Authorities and Community Health Management Teams were obtained prior to study implementation and regular feedback of study progress is given. Informed written consent is obtained from parents for case-control studies. Verbal consent is obtained for vital events surveillance and verbal autopsy. Ethical approval was provided by the Malawian National Health Sciences Research Committee and University of Malawi Research Ethics Committee, and by the University of Liverpool and London School of Hygiene & Tropical Medicine.

Discussion
Maintaining high quality real world surveillance at large scale is challenging [42]. We outline the strengths and weaknesses of our vaccine evaluation platform.

Strengths
Incorporating established large-scale population based surveillance, the program covers a very large population, spanning urban, peri-urban and rural settings. It has the capacity to determine vaccine effectiveness against mortality and morbidity in the context of high rates of malnutrition and HIV. Because the studies include rural and urban populations and span primary to referral level care, the results are likely to be relevant to other countries in sub-Saharan Africa or to low income countries elsewhere. Study sites have strong community engagement and acceptance, and have high quality track records in population epidemiology and clinical trials. Strong laboratory surveillance capacity has been well established at MLW for over a decade, allowing pneumococcal serotype and rotavirus genotype distributions to be described before and after respective vaccine introductions [3,22].
In contrast to vaccine evaluations elsewhere, studies in Malawi are occurring in a pneumococcal and rotavirus vaccine naïve population, so can determine changes in pneumococcal serotype and rotavirus genotype distribution for currently available vaccines. Using multiple sites and methods provides complementary and independent measures of vaccine effectiveness. Measures of effectiveness against specific (e.g. IPD, RV proven gastroenteritis) and less specific endpoints (e.g. radiological pneumonia, all-cause mortality) are of major public health importance, and address questions of value to policy makers. Nesting of multiple observational studies examining different endpoints and spanning several sites provides more robust evidence of impact than any single study could provide. Involvement of multiple agencies within the Ministry of Health and its NITAG ensures integration of studies within existing government programmes thus engendering strong community engagement and acceptability.

Challenges
Vaccine ascertainment is a major challenge (Table 4). Measurement of vaccination status depends predominantly on health passports, the availability of which may be biased [43]. We use multiple methods to obtain recorded documentation of vaccination including home visits and cross-checking against health centre vaccination registers. Retention of health passports among living vaccine age-eligible children at 1 year of age is high (93% in Mchinji and 85% in Karonga) [43]. There is however, a common practice in Malawi to bury the health passport with the deceased child. We obtain vaccination status as early as possible following schedule completion and cross check against clinic registers in an attempt to mitigate this problem.
Another challenge is determining vaccine effect in the setting of multiple interventions. Malawi has seen a decline in pneumococcal disease and in infant and child mortality over the past decade [3,20,44]. Attributing any observed reduction in morbidity or mortality to vaccine introduction, compared with the historical trend is difficult. Multiple government and non-government funded interventions have been implemented in Malawi. In Karonga for example, large scale sanitation efforts were implemented around the introduction of rotavirus vaccine. Such interventions, their scope and coverage affect the assessment of vaccine programme impact, and are difficult to adjust for in analysis. The study methods primarily determine concurrent individual level assessment of vaccine effectiveness by vaccine receipt, adjusting for individual and household exposure to communal interventions. Before and after analysis is also informative but attribution of any observed changes to vaccine alone is problematic.
Dealing adequately with confounders that increase the risk of disease outcomes among unvaccinated children independent of vaccine receipt is difficult. As population coverage rises the unvaccinated group becomes less representative of the general population. We measure socioeconomic and demographic covariates and adjust for these, but cannot deal with unmeasured confounders in the absence of randomisation. For incidence analysis we will conduct propensity scoring to help account for ascertainment bias arising from differential care-seeking behaviours, and adjust for prevalence of HIV, malnutrition and malaria [3,22,45].
Obtaining adequate sample size is a problem for some studies. The decline in IPD makes achieving recruitment targets in case-control designs difficult. This has led to the abandonment of the case-control design to evaluate indirect effects (Study 3) and increases the importance of ecological impact evaluation, but with the latter design attribution of any incidence reduction to vaccine must be cautious. Given our observed baseline mortality was lower than was originally anticipated based on published estimates [20], we will only be powered for high end decreases in all-cause mortality of greater magnitude than previously observed [6], increasing the importance of having independent measures to provide consistency. In addition, the introductions of PCV13 and RV1, were separated by 11 months. Reductions in all-cause mortality achieved by PCV13 will make detecting a benefit of RV1 more difficult, so cause-specific mortality must be used which introduces a level of uncertainty regarding the precision of causal determination by verbal autopsy. Although we will examine the impact of HIV on vaccine effectiveness, we are not adequately powered for definitive conclusions regarding this important issue.
Completeness and quality of reporting is a continual challenge. With over 1000 field workers, heterogeneity in data quality is inevitable. Supportive supervision, simple data collection instruments and automated performance target triggers are used to improve data reporting by key informants and rectify underperforming catchment areas. Long term sustainability of funding, which is currently borne through competitive research funding mechanisms, remains a challenge [46].

Interpretation
Maintaining large scale integrated multi-site epidemiological research of high quality is challenging, requiring ongoing investment in quality assurance of data collection. Reliance on existing data systems like health passports and vaccine registers is hampered by incomplete recording and by cultural factors, so measuring vaccine status requires early proactive effort if observation bias is to be minimised. Formally imputing vaccine status to deceased children is also subject to biases arising from our imputation modelling assumptions. Community engagement within existing socio-cultural systems is critical to the acceptability of surveillance. Using multimodal methods of analysis that include individual evaluation of effectiveness and ecological evaluation of impact is important, particularly for the detection of indirect vaccine effect. Integrating several strands of evidence with consistent results will provide greater confidence in surveillance findings for policy makers to act on. Other countries should of course consider their own epidemiologic settings when evaluating the generalisability of our results.
A long-term challenge facing this work and identified by the Global Framework for Immunization Monitoring and Surveillance is financial sustainability [46]. Establishing the early public health benefits following vaccine introduction produces data to advocate for broader vaccine use. Long term monitoring, though more prosaic, is no less important since it maintains the quality of the vaccination programme. By building this surveillance system collaboratively with the Malawi national immunization programme, utilising established data sources, training local health staff, and establishing locally feasible data management systems we have gone some way to meet the Global Framework requirements, but sources of continued funding will be required.

Conclusion
Despite the challenges, evaluating how effective new vaccines are in a real-world context is critical to encouraging vaccine introductions and sustaining vaccination programmes. Our large-scale surveillance system provides a platform for evaluating impact of PCV13 and RV1 on infant mortality and morbidity from IPD, radiologically confirmed pneumonia and severe gastroenteritis, and will determine cost-effectiveness in one of the world's poorest countries. The system is powerful and of high quality but faces considerable challenges, reflecting the real-world difficulties of conducting high quality large scale research in infrastructure-poor settings. Demonstrating an impact on mortality (due 2016) and morbidity (due 2015) will be important for the long term investment required to sustain vaccination programmes and to maximise their reach and delivery.