Rotavirus vaccine effectiveness in low-income settings: An evaluation of the test-negative design

Highlights • Rotavirus vaccines (RVs) have been recently introduced in low-income settings.• Accurate post-introduction monitoring of RV effectiveness is important.• The test-negative design (TND) is used to measure RV effectiveness.• The TND was evaluated using randomized trials of RV in sub-Saharan Africa and Asia.• The TND is an appropriate method to measure RV effectiveness in low-income settings.


Introduction
Globally, an estimated 200,000 deaths due to rotavirus diarrhea occur annually in children <5 years old, with a majority of the burden in low-income settings [1]. Starting in 2006, two rotavirus vaccines have been introduced worldwide; GlaxoSmithKline's live-attenuated human monovalent vaccine (Rotarix [RV1]) and Merck's live-attenuated pentavalent human-bovine reassortant vaccine (RotaTeq [RV5]). Large multi-site randomized controlled trials (RCTs) of RV1 and RV5 in low-income settings have demonstrated moderate vaccine efficacy against severe rotavirus gastroenteritis in the first year of life (VE: 51-64%) [2][3][4][5][6]. As of May 1, 2016, rotavirus vaccines have been introduced nationally in 38 Gavi-eligible countries. However, many high-burden countries have not introduced the vaccine and approximately 70% of the world's infants still do not have access to rotavirus vaccine [7]. Accurate post-introduction monitoring of effectiveness measures is important as results can influence the adoption of rotavirus vaccines in new areas and sustain support in countries where vaccines have been introduced.
Case-control studies are an efficient means to monitor effectiveness and provide confidence in vaccine performance. In lowincome settings, identifying community controls, either using a demographic surveillance system or sampling the community inperson, can be impractical and expensive. Hospital controls can be used to minimize bias due to healthcare seeking behavior. However, for rotavirus vaccine studies, careful consideration must be made to use hospital controls without diarrhea or any illness associated with vaccine-preventable diseases. The test-negative design traditionally-used control groups, while also limiting bias due to healthcare seeking behavior [8]. TND rotavirus vaccine studies enroll cases presenting to a medical facility for acute gastroenteritis and are rotavirus-positive using standard laboratory methods. Controls include those presenting to a medical facility with the same pre-defined case definition of acute gastroenteritis, but are rotavirus-negative. Both traditional case-control and testnegative study designs require rotavirus testing on infants presenting to the clinic with diarrhea to identify cases. The TND is efficient and cost-effective in that those testing-negative for rotavirus serve as the control group, instead of being excluded from the study.
The TND has been used extensively to measure annual influenza vaccine effectiveness [8,9]. Simulation experiments have validated the test-negative design for influenza vaccine under specific core assumptions: (1) vaccine has no effect on the incidence of noninfluenza pathogens, (2) a highly sensitive and specific laboratory test is used for pathogen detection, and (3) other sources of bias present in observational studies are minimized [8][9][10][11][12][13][14]. De Serres et al. validated the TND for influenza vaccine utilizing RCT databases to verify the accuracy and precision of TND estimates and to test the assumption that the vaccines had no effect on noninfluenza respiratory illness [15]. RCTs are appropriate to validate this design due to limited selection bias and confounding as a result of randomization and blinding, the use of standardized laboratory testing, and enhanced surveillance. Derived test-negative vaccine effectiveness estimates for influenza vaccines were almost identical to the original RCT vaccine efficacy estimates. Importantly, the vaccine coverage in the test-negative controls represented the vaccine coverage in the underlying study population, upholding the key assumption that the vaccine had no effect on non-influenza illness. Together, these results indicated the TND was a valid epidemiologic study design to measure influenza vaccine effectiveness [15].
The TND is being increasingly used to estimate rotavirus vaccine effectiveness in middle-and low-income settings due to its low cost and feasibility, [16][17][18][19][20][21][22][23][24][25][26] but little has been done to assess this epidemiologic study design in the context of rotavirus vaccine effectiveness in low-income settings. In the present analysis, RCT databases for RV1 and RV5 in sub-Saharan Africa and Asia were used to evaluate the TND.

Participants and study design
Databases from three multi-center, double-blind, individualrandomized, placebo-controlled, trials of rotavirus vaccines in sub-Saharan Africa and Asia were used [2][3][4][5][6]. Table 1 summarizes location, vaccine schedule, per-protocol population size, and surveillance type of the three RCTs.

RV1
This trial was conducted in South Africa and Malawi. Between 2005 and 2007, 4939 healthy infants aged 5-10 weeks were randomly assigned to one of three groups in a 1:1:1 ratio: two doses of RV1, three doses of RV1, or three doses of placebo. Gastroenteritis was defined as three or more loose or watery stools within 24 h. Clinical characteristics of each diarrheal episode were documented to define severity based on the Vesikari score [27]. Stool samples were tested for rotavirus using enzyme-linked immunosorbent assay (ELISA). The primary outcome was at least one episode of severe rotavirus gastroenteritis (Vesikari score P11). Vaccine efficacy was estimated during the period from two weeks after the last dose until the first year of age. Within each study site, a sub-cohort was followed into the second year of life. The mean age at the end of follow-up was 14 months and 19 months for South Africa and Malawi, respectively.

RV5
Two trials of RV5 were conducted in sub-Saharan Africa and Asia between 2007 and 2009. Both trials were conducted under similar protocols; however, the trials were powered and implemented separately. In sub-Saharan Africa, 5468 healthy infants were enrolled in Ghana, Kenya, and Mali. In Asia, 2036 healthy infants were enrolled in Bangladesh and Vietnam. Infants aged 4-12 weeks were randomly assigned to one of two groups in a 1:1 ratio: three doses of RV5 or three doses of placebo. As in the RV1 trial, severe rotavirus gastroenteritis was defined based on a positive ELISA laboratory result and Vesikari score P11. Vaccine efficacy was estimated during the period from two weeks after the last dose until the end of follow-up (March 31, 2009). The mean age at the end of follow-up was 20 months and 19 months for sub-Saharan Africa and Asia, respectively.
For the purposes of this analysis, participants with an episode of severe diarrhea meeting the pre-defined case definition and with an available ELISA test result were categorized as a case if the test was positive for rotavirus or a control if the test was negative for rotavirus. Continuous diarrheal surveillance during the study period allowed for the identification of multiple diarrheal episodes for each participant. A participant was defined as a case if at least one severe rotavirus-positive diarrheal episode occurred during follow-up. A participant was defined as a control if at least one severe rotavirus-negative diarrheal episode occurred during follow-up and the participant had no severe rotavirus-positive episodes.

Statistical analysis
Logistic regression was used to estimate the relative odds and associated 95% confidence intervals (CI) of severe rotaviruspositive diarrhea compared to severe rotavirus-negative diarrhea by vaccine and placebo status. TND vaccine effectiveness (VE-TND) was defined as (1 À Odds Ratio) Â 100. The relative percent difference between VE-TND and the original RCT vaccine efficacy (VE-RCT) was calculated. To estimate the influence of vaccine on rotavirus-negative diarrhea (VE-NEG), the relative risk of severe rotavirus-negative diarrhea in the vaccine group compared to the placebo group with exact 95% CIs were calculated. VE-NEG was defined as (1 À Relative Risk) Â 100.
The analysis was based on the per-protocol participant populations. The primary analysis of each RCT, combining country-level estimates, was replicated. Additionally, analyses were stratified by country. The RV1 trial was powered to estimate vaccine efficacy for two and three doses separately, therefore analyses were replicated by these dosing combinations. Each trial included diarrheal surveillance on all or a subset of participants into the second year of life. Analyses were conducted for the complete follow-up period (<2 years of age) and separately for diarrheal episodes identified during the first (<1 years of age) and second (1-<2 years of age) years of life. Analyses conducted within the second year of life did not exclude participants with diarrheal episodes during the first year of life. For the RV1 trial, different methods of enrollment into the second year were used for each country; therefore, analyses for the second year of life and complete follow-up were conducted separately for South Africa and Malawi.
In practice, differences in age and time at presentation between cases and test-negative controls are controlled by matching or by adjusting analyses by both month and year of birth and month and year of presentation. Additional analyses were restricted by rotavirus season in Ghana and Mali due to the observed seasonality in these regions. To replicate the primary efficacy results of the RV5 trial, diarrheal episodes identified during the rotavirus seasons in Ghana and Mali were combined with the year-round diarrheal episodes in Kenya to estimate country-combined estimates.
Analyses were completed using STATA version 14 (Stata Corporation, College Station, TX, USA) and the SAS Clinical Trial Data Transparency software system through the online GSK portal (SAS Institute Inc., Cary, NC, USA). Table 2 shows the derived test-negative vaccine effectiveness estimates (VE-TND), the original RCT vaccine efficacy estimates (VE-RCT), and the relative percent difference between these estimates in the RV1 trial. During the first year of life (<1 years of age), the country-combined and country-specific efficacy and effectiveness estimates for all doses were similar (two or three doses combined in South Africa and Malawi: VE-TND: 58.2% [95% CI: 35.5-72.9]; VE-RCT: 61.2% [95%CI: 44.0-73.2]; À4.9% relative difference). The sub-cohort in South Africa followed over two rotavirus seasons yielded a low sample of rotavirus-positive cases resulting in less robust estimates during both the second year of life (1À<2 years of age) and using complete follow-up (<2 years of age). The sub-cohort in Malawi followed over two rotavirus seasons resulted in VE-TND and VE-RCT estimates which were not meaningfully different during any periods of follow-up.

Results
VE-TND and VE-RCT estimates for the RV5 trial in sub-Saharan Africa are shown in Table 3 In this trial, the rotavirus season in Ghana occurred between January and March. After restricting the analysis to diarrheal episodes during this time period (Table 3) Fig. 1 summarizes the VE-TND and VE-RCT estimates and overlapping 95% CIs for all RCTs during the first year of life and using the complete follow-up (<2 years of age). Table 5 shows both RV1 and RV5 vaccine efficacy against severe rotavirus-negative diarrhea (VE-NEG). In the RV1 trial, the magnitude of VE-NEG was greatest in South Africa during the second year of follow-up with three doses of vaccine (54.0% [95%CI: 7.1-77.2]), while all other age and dose combinations had a low VE-NEG. In the RV5 trial in sub-Saharan Africa, the magnitude of the VE-NEG was greatest in Ghana during the second year of life (1-<2 years of age: À50.5 [95%CI: À170.9-14.9]), and using complete followup (<2 years of age: À49.1 [95%CI: À117.9 to À2.7]), compared to other settings. The country-combined VE-NEG was statistically significant using complete follow-up. After restricting the analysis to the rotavirus season using complete follow-up, the VE-NEG decreased in all RV5 settings with no statistically significant estimates. In the RV5 trial in Asia, the magnitudes of the countrycombined VE-NEG estimates were low (<20%) during all followup periods. Due to the low magnitude of each VE-NEG, the ratio of vaccine: placebo test-negative controls replicated the 2:1 or 1:1 randomized vaccine coverage of the underlying study population in most study settings.

Discussion
Overall, the results from the TND analysis for RV1 and RV5 in sub-Saharan Africa and Asia were similar to primary efficacy results. The heterogeneity of vaccine effectiveness estimates between countries was also observed. In countries with a marked rotavirus season, estimates were not meaningfully different after restricting analyses to these time periods. We also demonstrated that RV1 and RV5 had no effect on severe rotavirus-negative diarrhea, a key assumption of the TND.
VE-TND during the first year of life was comparable to VE-RCT for all three trials. The control group accurately represented the vaccine coverage of the underlying study population. For example, in the RV1 trial the ratio of vaccinated (two doses) controls to placebo controls is nearly 1:1 (93:91), the randomized vaccine: placebo ratio. In all trials, VE-TND estimates during the second year of life were largely limited by the low number of diarrheal episodes and rotavirus-positive cases identified. Primary VE-RCTs for the second year of life were not statistically significant and this was replicated in the VE-TND results. The sustained effect of rotavirus vaccine during the second year of life remains unclear [3,28].
In the analysis using complete follow-up (<2 years of age), the VE-TND was similar to the VE-RCT in RV1 and RV5-Asia. Differences between the estimates were demonstrated in the RV5 trial in sub-Saharan Africa for country-combined and country-specific estimates. Ghana and Mali have distinct rotavirus seasons. Analyses were restricted to the rotavirus season in order to obtain time-matched controls. This strategy better emulates the practice of incidence density sampling and provides results that are closer  to the relative risk (and in turn VE = 1-Relative Risk) derived from an RCT. These results support the importance of temporally matched controls or accounting for timing of birth and case presentation in the TND to obtain accurate vaccine effectiveness estimates. Generally, less robust VE-TND and VE-RCT estimates in Mali are in part due to changes in surveillance after the first year of the study, which initially missed most of the rotavirus season and yielded a low number of rotavirus cases. Surveillance and community engagement increased during the second year of the trial to increase case capture [5]. Notably, during the first year the ratio of vaccinated controls to placebo controls is almost exactly 1:1 (48:49), indicating the surveillance appropriately identified the underlying vaccine coverage in test-negative controls during this time.
Further evidence of the accuracy of the test-negative control group is demonstrated in rotavirus vaccine case-control studies using multiple control groups in sub-Saharan Africa and Latin America [17,19,20,23,29]. In Malawi, RV1 effectiveness for severe rotavirus-positive diarrhea during the first year of life was similar using test-negative controls and community controls (VE-TND: 68%, VE-Community controls: 68%) [17]. In South Africa, RV1 effectiveness for hospital admission with rotavirus-positive diarrhea in children <2 years old was comparable using test-negative controls and hospitalized controls (VE-TND: 57%, VE-Hospital controls: 63%) [20]. In Bolivia, RV1 effectiveness for severe rotaviruspositive diarrhea during the first year of life was moderately different using test-negative controls and hospitalized controls (VE-TND: 66%, VE-Hospital controls: 78%) [29]. In Nicaragua, RV5 effectiveness for severe rotavirus-positive diarrhea during the first year of life was slightly different using test-negative controls and a combined group of hospitalized and community controls (VE-TND: 70%, VE-Hospital/Community controls: 83%). In children >1 years old, the difference in estimates was more substantial (VE-TND: 33%, VE-Hospital/Community controls:70%) [19]. Authors suggest the combined control group differed from the test-negative and test-positive participants, likely due to healthcare seeking behaviors. In Guatemala RV1 and RV5 effectiveness against a rotavirus-positive diarrheal episode resulting in a hospital visit was moderately different using test-negative controls and hospitalized controls (VE-TND: 52%, VE-Hospital controls: 74%) [23]. All studies observed vaccine effectiveness results similar to efficacy estimates of RCTs conducted in Latin America.
Importantly, while the TND can be valuable, this study design is susceptible to biases present in all observational studies including selection bias, confounding, and misclassification of vaccine and rotavirus status [14]. In these RCTs we are highly confident of accurate vaccine ascertainment, but this can be problematic in the field with missing documentation, unreliable parental recall, and the added complexities of multiple doses [30]. Additionally, the  sensitivity and specificity of the test to identify the etiologic pathogen is especially important when using the TND. A test with low specificity influences results more substantially than a test with low sensitivity [10]. All RCTs and case-control studies used ELISA for rotavirus detection. While RT-PCR is the gold standard, ELISA has high sensitivity (75-82%) and specificity (100%) to identify rotavirus [31]. A simulation study comparing true and estimated TND vaccine effectiveness results based on varying rotavirus test characteristics and attack rates demonstrated minimal bias with the currently used ELISA [32]. This analysis evaluated the use of the TND to estimate rotavirus vaccine effectiveness in low-income settings. Three separate rotavirus vaccine trials, testing two vaccines in seven countries, showed TND vaccine effectiveness estimates were nearly identical to the primary efficacy estimates of the original RCTs. The key assumption of the TND, the vaccine has no impact on rotavirus-negative diarrhea, was also upheld. This study supports the testnegative design as an appropriate method to measure rotavirus vaccine effectiveness in low-income settings. Gates Foundation or the National Institutes of Health. Funders of the study had no role in study design, data collection, data analysis, data interpretation, writing of the report, or the decision to submit the article for publication.

Contributors
All authors contributed to the design of the study. LMS and JCV obtained the data. LMS analyzed the data. All authors interpreted the data. LMS wrote the manuscript and all authors revised the manuscript. All authors approved the final version of the manuscript.