Effectiveness of Rotarix® vaccine in Africa in the first decade of progressive introduction, 2009-2019: systematic review and meta-analysis [version 1; peer review: 1 approved with reservations]

Background: Randomized controlled trials of licensed oral rotavirus group A (RVA) vaccines, indicated lower efficacy in developing countries compared to developed countries. We investigated the pooled effectiveness of Rotarix® in Africa in 2019, a decade since progressive introduction began in 2009. Methods: A systematic search was conducted in PubMed to identify studies that investigated the effectiveness of routine RVA vaccination in an African country between 2009 and 2019. A meta-analysis was undertaken to estimate pooled effectiveness of the full-dose versus partial-dose of Rotarix® (RV1) vaccine and in different age groups. Pooled odds ratios were estimated using random effects model and the risk of bias assessed using Newcastle-Ottawa scale. The quality of the evidence was assessed using GRADE. Results: By December 2019, 39 (72%) countries in Africa had introduced RVA vaccination, of which 34 were using RV1. Thirteen eligible studies from eight countries were included in meta-analysis for vaccine effectiveness (VE) of RVA by vaccine dosage (full or partial) and age categories. Pooled RV1 VE against RVA associated hospitalizations was 44% (95% confidence interval (CI) 28-57%) for partial dose versus 58% (95% CI 50-65%) for full dose. VE was 61% (95% CI 50-69%), 55% (95% CI 32-71%), 56% (95% CI 43-67%), and 61% (95% CI 42-73%) for children aged <12 months, 12-23 months, <24 months and 12-59 months, respectively. Conclusion: RV1 vaccine use has resulted in a significant reduction in severe diarrhoea in African children and its VE is close to the efficacy Open Peer Review


Introduction
Globally, rotavirus group A (RVA) is a leading cause of severe dehydrating acute diarrhoea in children aged <5 years 1 . In 2016, approximately 117 million episodes of rotavirus-associated diarrhoea occurred in sub-Saharan Africa, ~104,000 of which were fatal 2 . Rotavirus vaccination programmes are considered the most effective control measure for RVA disease 3,4 and six oral vaccines (RotaTeq® (RV5), Rotarix® (RV1), Rotavac®, Rotavin-mi®, Lanzhou Lamb and Rotasiil®) have been licensed 5 . In 2009, the World Health Organization recommended inclusion of two licensed RVA vaccines (Rotarix®, GlaxoSmithKline Biologicals, Belgium; and RotaTeq®, Merck, USA) into routine national immunization programmes (NIP) of all countries 6 . By July 2020, 74% of African countries (40 out of 54) had introduced an RVA vaccine into their NIP compared to the global tally of 107 out of 194 countries (55%) 7 . Of the 40 African countries with the RVA vaccine, 35 (88%) were using the RV1 vaccine.
Randomised controlled trials (RCTs) investigating the efficacy of oral RVA vaccines in sub-Saharan Africa showed a modest performance (50-80% efficacy against severe disease) compared to results from industrialized countries (90-100% efficacy) 8,9 . Despite this discrepancy, use of RVA vaccines in developing countries was encouraged on the basis of the expected absolute impact on the high RVA disease burden in low-income setting 10 . Post-vaccine introduction, a number of African countries have reported on RVA vaccine impact and effectiveness against rotavirus gastroenteritis (RVGE) and all-cause diarrhoea related hospitalisations 11 . There have been expert reviews discussing the impact of RVA vaccine in African countries 12-14 and systematic review and meta-analysis conducted focusing on the prevalence of rotavirus infections pre-and post-vaccine introduction 15 . In this paper, we present a systematic review and meta-analysis of vaccine effectiveness of rotavirus vaccination programmes in Africa focusing on partial dose versus full dose and effectiveness stratified by age categories.

Systematic search
We conducted a systematic search in PUBMED database for articles on research conducted in African populations from January 2009 to December 2019 focusing on the rotavirus vaccination programme and adhered to PRISMA guidelines (Extended data: Supplementary File One, Supplementary Table 1) 16 . Publications were identified using combinations of the following key search terms: "Rotavirus", "effectiveness", "success", "impact", "effect", "potency", "performance", "vaccine", "Rotarix", "Rotateq" and names of all 54 African countries. We restricted our search to articles published in English (see Extended data: Supplementary File One, Supplementary Text 1 for details) 16 . Two reviewers screened the outputs identified from the searches for appropriate articles and from references of the relevant published articles to identify additional articles for possible inclusion into the analysis. The final included articles were based on agreement between the two reviewers. A third reviewer resolved any discrepancies. Information on RVA vaccine introduction status for each country and impact evaluation were inferred from VIEW-hub 17 .

Inclusion criteria and outcomes
This analysis focuses on the Rotarix® (RV1) Vaccine, which is given to infants as two doses at 6 and 10 weeks of life. We aimed to include articles published from any African country that administers RV1 vaccine as part of the NIP. Observational studies (case-control) reporting on the effectiveness of RVA vaccine among children aged <5 years in their country, on RVGE or other acute gastroenteritis (AGE) hospitalization between 2009 and 2019 were included. Outcomes of interest included effectiveness RVA vaccine against hospitalization from RVGE for full dose, partial dose and, effectiveness stratified by age categories. Randomized controlled trials, review articles, editorials and conference papers were excluded from this analysis (Extended data: Supplementary File One, Supplementary Text 2) 16 .

Data extraction
The following details were extracted from the eligible studies; study design, sample size, country, vaccine type, age groups, cases vaccinated, controls vaccinated, reported vaccine effectiveness and 95% confidence intervals (CIs). Extracted data was entered into data collection forms created in Microsoft Excel (Extended data: Supplementary File Two) 16 .

Assessment of risk of bias
The Newcastle-Ottawa scale (NOS) was used to assess the risk of bias (ROB) among the case-control studies 18 . The NOS was used to evaluate the selection of participants, comparability of study groups, and the ascertainment of exposure or outcome of interest.
A study was assigned a maximum of 9 points based on selection (4 stars), comparability (2 stars) and exposure (3 stars), for a maximum of 9 points, by using a star allocation scheme according to the coding manual developed by collaboration between Universities of Newcastle and Ottawa 19 . Studies scoring zero in any of the categories were classified as having high ROB. Studies scoring 1 point in any of the categories were classified as having moderate ROB, and those scoring 2 points or more in all categories were classified as having low ROB.

Data management and analysis
Meta-analysis was stratified (full and partial dose) for RV1 vaccine effectiveness and by age categories (<12 months, 12-23 months, <24 months, and 12-59 months). We used study reported vaccine effectiveness (VE) estimates to obtain the respective 1 -( 100) OR VE = and the respective log transformed odds ratio (OR). A random effects model was used to estimate the pooled VE while accounting for variations of the true effect size due to varying geographical settings of the studies.
Heterogeneity was assessed by the chi-squared test for heterogeneity and quantified by I 2 index [(Q-df) / Q x 100 %] where Q is the Cochran's heterogeneity statistics and the degrees of freedom (df). I 2 values of 25-49%, 50-74% and >=75% were categorized as low, moderate, and high heterogeneity respectively 20 . Forest plot was used to present the pooled ORs with their corresponding 95% CI. To check for publication bias, funnel plot was used, and Eggers test employed to assess funnel plot asymmetry. All statistical analyses were conducted using STATA version 15.1 (StataCorp College Station, Texas), and where applicable admetan package was used 21 .
Quality of the evidence Two reviewers independently assessed the quality of evidence using the Grading of Recommendation, Assessment, Development, and Evaluation (GRADE) approach 22 . Quality of evidence was graded as high, moderate, low, or very low. GRADE starts with a baseline rating of 'high quality of evidence" for RCTs, and "low quality of evidence" for non-RCTs 23 . Given that only observational studies were included in this review we assessed the quality of evidence starting-off as "low quality" and downgraded or upgraded accordingly. Reasons for downgrading included high risk of bias, inconsistency or heterogeneity, indirectness of the findings, imprecision of the point estimates, and publication bias. The quality of evidence was upgraded if data showed a large effect, a dose-response effect, or if all the plausible residual confounding would reduce the demonstrated effect or would suggest a spurious effect if no effect was observed.

Search outcome
A total of 324 published articles were identified based on our defined search criteria. Of these, 13 met our inclusion criteria for the meta-analysis ( Figure 1; Table 1 and Table 2). All the 13 studies reported the full dose vaccine effectiveness and five studies also reported on effectiveness of partial dose of RV1 vaccine as well. The included studies originated from eight countries. The large majority of studies were excluded from the analysis after screening the title and abstract (n=259). Fifty-two studies were excluded after full-text screening. Of these, 28 were due to evaluation of the impact of RVA vaccines, 13 were systematic reviews, four were rotavirus symposium report, and rotavirus strain distribution respectively, two were evaluating the effectiveness of RVA pentavalent vaccine (RotaTeq®), and one was an informative interview ( Figure 1; see Extended data: Supplementary File One, Supplementary Text 2 for details) 16 .
Risk of bias in the included studies ROB in the included studies was assessed, and none was found to have a high risk of bias (Table 3). The assessment using NOS was based on selection, comparability, and exposure. With regard to selection; most studies had adequate selection and representativeness of cases, controls were selected within the same population as cases, and it was difficult to ascertain if the history of rotavirus among the control was considered. With regard to comparability; confounding was controlled through adjusting for age, date of birth, and month and year of admission in the analysis for the majority of the studies. Lastly, exposure was ascertained through the vaccination cards held by a guardian or parent. It was difficult to ascertain whether the non-response rate was considered in the analysis of most studies (Table 3; see Extended data: Supplementary File Three for details) 16 .

Sensitivity analysis
Sensitivity analysis was carried out by excluding Armah et al. 2016 27 in our meta-analysis of full and partial dose VE. This study reported full dose vaccine effectiveness of 18% (95% CI -81-63%). This was due to high vaccine coverage (93-100%) immediately after its introduction in Ghana making it difficult to arrive at robust VE estimate 27 . VE estimate for full dose by excluding this study in our meta-analysis was 58% (95% CI 50-65%) (Extended data: Supplementary File One, Supplementary Figure 1) 16 . More so, sensitivity analysis was carried out to ascertain the source of high heterogeneity

Quality of evidence
Pooled VE for full and partial dose had moderate quality of evidence. We started at the low quality of evidence because our pooled effects were based on case-control studies (observational studies). No considerable bias was detected using the NOS, all studies were conducted in Africa and directly address review questions, and no heterogeneity or imprecision which was observed to warrant downgrading. However, the evidence  was upgraded to moderate quality since the magnitude of the effect was high and consistent throughout the included studies (Table 4).
Our effect estimate based on age categories had low quality of evidence. The evidence was downgraded by one for imprecision due to the few numbers studied used within the different age categories. However, we upgraded the evidence by one since the reported estimates are consistent within the different age categories. No further adjustment was made as no considerable bias was detected by the NOS and all studies were conducted in Africa and directly addressed the review question (Table 5). Evidently, this estimates of VE are higher than our findings in Africa, hence portraying a similar scenario to the pre-licensure evaluation of rotavirus vaccine efficacy whereby efficacy in high-income countries were higher than low-income countries 9 . We have also shown evidence of protection against RVA-associated severe diarrhoea beyond the first year of life.

Majority of African countries have introduced the Rotarix
This analysis had some limitations. Primarily, the number of studies reporting effectiveness of RV1 vaccine was still low compared to the number of countries that have introduced RV1. We only used data from the limited number of studies that have been published to date. Including more studies in a future meta-analysis will improve our certainty of the pooled VE estimates from the African continent both for the different age categories and for partial-and full-dose assessment.
In conclusion, we show that RV1 vaccine effectiveness is substantial in Africa and is occurring within the range of efficacy findings observed in clinical trials. The pooled vaccine effectiveness was lower with a partial dose compared to full dose, thus increased coverage should be encouraged to reap the full benefits of this vaccine. Although the quality of evidence in the age-category based analysis was lower, the data so far appear to support the notion that VE of RV1 is high beyond the first year of life in African children.  Table 3. 3 There was no statistical heterogeneity (I 2 = 0%), there was also low methodological heterogeneity given that all included studies used similar study design. 4 The studies were all conducted in African countries and directly address the review question. 5 We did not downgrade for imprecision although some studies had wide confidence intervals. We conducted sensitivity analysis by removing Beres et al. 2016 (Extended data: Supplementary File One, Supplementary Figure 3) 16 , which had widest CI was removed from the meta-analysis and concluded it did not change the pooled estimate. 6 The magnitude of effect was high consistently throughout all included studies. Quality of evidence was upgraded by 1. 2 No considerable risk of bias was detected using the Newcastle-Ottawa scale (NOS), see Table 3. 3 There was statistical heterogeneity (Figure 4). We conducted sensitivity analysis by dropping Mujuru et al. 2019 from analysis because the author stated that VE estimate for this aged group lacked precision and was non-significant. No heterogeneity was observed after dropping this study. There was also low methodological heterogeneity given that all included studies used similar study design. 4 The studies were all conducted in African countries and directly address the review question. 5 We downgraded for imprecision by 1 due to small number of studies used in some groups, see Figure 3 6 The magnitude of effect was high consistently throughout all included studies. Quality of evidence was upgraded by 1. Nickson Murunga and collaborators present a comprehensive review and meta-analysis, pooling data from identified studies to determine effectiveness of RV1 (Rotarix®) in Africa during the last 10 years.

Data availability
The manuscript is well-written and precise (with some grammar mistakes that can be reviewed) with an adequate description of the search strategy and analysis process, although with the characteristic technical descriptions that requires expertise for appropriate evaluation. The search leads to a rather small (unfortunately not more data is available) but reasonably robust number of overall participants. The study should be reviewed by an expert in meta-analysis methods to assure that the methods used in this study are correct.

Specific comments: Introduction
Authors should explain and detail differences between terms "effectiveness" and "impact". As mentioned in background, RVA vaccine implementation in Africa was based on expected impact on disease, despite clinical trials showed lower efficacy compared to industrialized countries. Studies published post vaccine introduction are focused on its impact and/or effectiveness, and this review focuses only in those studies assessing effectiveness. It is possible to assume that "impact" refers to overall effect of vaccine in a population (which includes vaccinated and non-vaccinated subjects) and "effectiveness" refers to direct effect of vaccine (only in vaccinated subjects). However, in order to be understandable to general scientific community authors should specify this difference.

Methods:
The fact that the term "impact" was included in the search strategy followed by exclusion of ○ 28 studies focusing on impact of vaccine on diarrhea hospitalization and diarrhea associated mortality is unclear.
Authors should specify if "partial dose" refers to 1 dose of Rotarix, and "complete dose" to 2 doses in all studies included in this review.

○
There is no mention of diagnostic tests used to identify rotavirus infection in the different studies. As sensitivity and specificity differs between tests, this variable may affect effectiveness results in different studies.

Results
: Table 1 and 2 are excessively redundant (they can be merged).

Discussion:
More caution in the full dose vs partial dose conclusion as there is no significant difference.

○
Authors compare results of this meta-analysis with a study performed in Latin America which used both hospital and community controls. This lead to ask if all studies included in this meta-analysis used only hospitalized controls, and if so, was it because studies from a non-hospital setting were not found in the literature search?

○
Evidently, this estimates, should be these estimates.
○ Are the rationale for, and objectives of, the Systematic Review clearly stated? Yes

Are sufficient details of the methods and analysis provided to allow replication by others? Yes
Is the statistical analysis and its interpretation appropriate? I cannot comment. A qualified statistician is required.

Are the conclusions drawn adequately supported by the results presented in the review? Yes
Competing Interests: No competing interests were disclosed.
We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above. version.

Results Section
5. Table 1 and 2 are excessively redundant (they can be merged).

Authors Response:
We have merged the two tables. Included in our revised manuscript.
6. Table 3 is not required; it can be reduced to one sentence highlighting the only different study (Abeid et al.).
Authors Response: Thank you for the observation. This has been amended in our revised manuscript.
7. The reasoning provided for the sensitivity analysis are unclear; is this standard methodology?
Authors Response: Yes. This is a standard methodology. Sensitivity analysis is one of the strategies for addressing heterogeneity see https://training.cochrane.org/handbook/current/chapter-10#section-10-11.
8. Tables 4 and 5 are somewhat difficult to understand, should be better explained.
Authors Response: Thanks for this observation. More explanation is provided in our revised manuscript.

Discussion:
9. More caution in the full dose vs partial dose conclusion as there is no significant difference.
Authors Response: This has been rephrased in our revised manuscript.
10. Authors compare results of this meta-analysis with a study performed in Latin America which used both hospital and community controls. This lead to ask if all studies included in this meta-analysis used only hospitalized controls, and if so, was it because studies from a non-hospital setting were not found in the literature search?
Authors Response: Thank you for noting this. From our literature search, only one study from Malawi (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4374102/) used both hospital and community control. A similar magnitude of vaccine effectiveness was established for both controls. Therefore, only hospital control was included in this meta-analysis.