Agreement between ELISA and plaque reduction neutralisation assay in Detection of respiratory syncytial virus specific antibodies in a birth Cohort from Kilifi, coastal Kenya.

Background: Severe disease associated with respiratory syncytial virus (RSV) infection occurs predominantly among infants under 6 months of age. Vaccines for prevention are in clinical development. Assessment of the vaccine effectiveness in large epidemiological studies requires serological assays which are rapid, economical and standardised between laboratories. The objective of this study was to assess the agreement between two enzyme linked immunosorbent assays (ELISA) and the plaque reduction neutralisation test (PRNT) in quantifying RSV specific antibodies. Methods: Archived sera from 99 participants of the Kilifi Birth Cohort (KBC) study (conducted 2002-2007) were screened for RSV antibodies using 3 methods: ELISA using crude RSV lysate as antigen, a commercial RSV immunoglobulin G (IgG) ELISA kit from IBL International GmbH, and PRNT. Pearson correlation, Bland-Altman plots and regression methods were used in analysis. Results: There was high positive correlation between the IBL RSV IgG ELISA and PRNT antibodies (Pearson r=0.75), and moderate positive correlation between the crude RSV lysate IgG ELISA and PRNT antibodies (r= 0.61). Crude RSV lysate IgG ELISA showed a wider 95% limit of agreement (-1.866, 6.157) with PRNT compared to the IBL RSV IgG ELISA (1.392, 7.595). Mean PRNT titres were estimated within a width of 4.8 log 2PRNT and 5.6 log 2PRNT at 95% prediction interval by IBL RSV IgG and crude RSV lysate IgG ELISA, respectively. Conclusion: Although, the IBL RSV IgG ELISA is observed to provide a reasonable correlate for PRNT assay in detecting RSV specific antibodies, it does not provide an accurate prediction for neutralizing antibody levels. An RSV neutralising antibody level is likely to fall within 2.4 fold higher and 2.4 fold lower than the true value if IBL RSV IgG ELISA is used to replace PRNT assay. The utility of an ELISA assay in vaccine studies should be assessed independent of the PRNT method.


Introduction
Respiratory syncytial virus (RSV) is an important cause of annual epidemics of bronchiolitis and pneumonia in children less than five years of age worldwide [1][2][3][4][5] . The most severe disease occurs among infants under 6 months of age 6 , making this group the principal target for RSV disease prevention.
To date there are no licensed vaccines for RSV. However, strategies to protect the young infant through immunisation are underway. One approach showing particular promise is to protect the infant over the first few months of life by maternal vaccination. A maternal RSV vaccine which is based on a post-fusion F protein nanoparticle design 7 , is now undergoing phase 3 clinical trials (NCT02624947). Standardised serological assays are needed if this or other vaccines show good efficacy as they go to effectiveness phase 3b/4 clinical trials.
The plaque reduction neutralisation test (PRNT) has been the preferred technique by most studies to determine the correlates of protective immunity to RSV [8][9][10][11][12][13] . This is because the assay determines the level of antibodies present that are presumed to be functional in vivo 14 . For instance, a pseudovirion-based neutralisation assay was found to detect all neutralizing antibodies that directly arrest the virus and thus to provide protection against infection with Human papilloma virus 15 . A study by Piedra et al. showed that serum antibody titres correlate with protection 8 ; a finding also supported by Stansbelle et al. who showed that neutralisation antibody titres were inversely associated with RSV hospitalization in infants <6months 16 . Hence PRNT is considered the most appropriate serological assay for use in quantifying protective immunity correlates to RSV.
Although the PRNT is considered the "Gold standard" method for quantifying the level of RSV specific antibodies, it has several limitations: (i) PRNT antibody titres are currently not internationally standardised and therefore it is difficult to compare results across different studies, though work is under way to resolve this 17 ; (ii) the assay procedure is cumbersome -taking four to six days (according to method) to complete; and, (iii) the assay is costly in terms of laboratory technologist time. Hence, the PRNT is not ideal for large sero-epidemiological studies or large-scale vaccine trials.
Due to these limitations, it would be beneficial to adopt a serological assay which is quicker, cheaper and can be easily standardized, for use in large vaccine studies. An ELISA method is an option. However, the method measures epitopes of the target antigen and does not discriminate antibodies specific to neutralizing sites. Presently, there are two RSV specific IgG ELISA assays in our laboratory: an ELISA which uses crude RSV A2 culture lysate as antigen (Crude ELISA), and a commercial ELISA (IBL ELISA) which uses 96 well micro titre ELISA plates pre-coated with RSV F protein and RSV group A cell extract as antigen. We examined the agreement between ELISA assays and the PRNT in quantifying RSV specific antibodies using serum samples from a birth cohort in Kilifi, a coastal part of Kenya.

Study site and participants
This study was conducted in Kilifi, within the Kilifi Health and Demographic Surveillance System (KHDSS) in the coastal part of Kenya 18-20 . Archived serum and plasma samples were utilised arising from the Kilifi Birth Cohort (KBC) study conducted between 2002 and 2007, by the Kenya Medical Research Institute-Wellcome Trust Research Programme (KEMRI-WTRP). The KBC study was an observational study to investigate susceptibility to invasive pneumococcal disease among children. In this study, newborn participants numbering around 6000, recruited at the maternity ward or the child health clinic of Kilifi County Hospital (KCH), were followed for a 2-year period with blood samples collected at birth and then every 3 months. Samples were stored as plasma (for cord bloods) or serum (follow-up samples) at -80°C. Details of the birth cohort study are described elsewhere 10,18,19 .
In the present study, a random sample set of 100 individuals from the KBC participants recruited from the KCH maternity ward, were selected from the study database. Description of the levels of maternal RSV specific neutralizing antibodies from blood samples of the 100 participants were provided in a previous publication 10 .

Ethical considerations
All parents and guardians gave written consent to have their children participate in the KBC study for storage of blood samples for use in future research. The use of the archived sample set was approved by the KEMRI-Scientific and Ethics Review Unit (SERU# 2307).

Laboratory procedures
Archived plasma/serum samples from the KBC study which had been stored at -80°C were retrieved and screened using 3 serological assay techniques: (i) a PRNT (ii) an IgG ELISA assay using crude RSV lysate as antigen (crude ELISA) and (iii) an RSV IgG ELISA using a commercial ELISA kit (IBL ELISA).

Plaque reduction neutralisation test (PRNT).
The PRNT procedure for determining the titre of RSV neutralising antibodies has been described previously 9 . The method incorporated a step in which serum samples were incubated at 56°C in a water bath for 30 minutes to inactivate complement cascade proteins. Each serum sample was repeatedly diluted 2-fold over ten consecutive dilutions and mixed with an equal volume of 50 plaque forming units (pfu) of RSV A2 virus (RSVA2 and Hep2 cells were a kind donation from Dr. Patricia Cane while she worked at the Health Protection Agency, UK). The virus-serum mixture (50µl per well) was dispensed over a confluent monolayer of Hep2 cells in a 96 well culture plate, incubated at 37°C for 1 hour and then underwent 4-hour cycles of rotation on an angled (about 30°) rotating platform (about 40 rev/minute) for 10 minutes and incubation in a 37°C CO 2 incubator for 30 minutes. The plate was then incubated for 48 hours in a 37°C CO 2 incubator. Fixation of cells was done by the addition of 100µl of fixation reagent (30% methanol+70% acetone). Plaques were detected by addition of a primary antibody (RSV F protein mouse monoclonal-BIO-RAD, Catalogue# MCA490) solution diluted 1:500 in PBS with 2 hours incubation at 37°C, followed by an addition of a 100µl/well of an anti-mouse HRP-conjugated secondary antibody (170-5047 Immun-Star Goat Anti-Mouse (GAM)-IgG (H/L) polyclonal antibody HRP-BIORAD) solution diluted 1:1000 in Phosphate Buffered Saline (PBS) with 1 hour incubation at room temperature. After each step, plates were washed manually three times using 200µl/well PBS buffer. Plaques were visualised by addition of 100µl/well detection reagent. This consisted of 16 µl of hydrogen peroxide and 0.6ml of 3-amino-9-ethlycarbazole 3.3mg/ml solution (20mg 3-amino-9-ethlycarbazole tablet were dissolved in 6.06ml of dimethyl sulphoxide (DMSO) to give a 3.3mg/ml solution) to 10ml of 20mM sodium acetate solution (pH 5.0-5.5). Reading and counting of the brown-stained RSV micro-plaques was done using an ELISpot reader (Autoimmun Diagnostika GmbH, Germany).
The dilution of a test serum sample required to induce 50% neutralization of a known titration of RSV A2 virus was determined using the Spearman Karber method 9 . In addition, a panel of control samples from BEI Resources (BEI RSV Reference panel catalogue #NR-32832) was included in each batch of the PRNT assay to monitor reproducibility of the assay results and deterioration of the antibodies used. Results obtained from screening of the BEI samples were compared with PRNT values of the samples as previously tested in BEI resources laboratories.
Crude RSV lysate IgG ELISA (Crude ELISA). Blood samples were tested for antibody concentration with an IgG based ELISA method using crude virus extract from lab-adapted RSV A2 culture following a local standard operating procedure 21 . The crude virus RSV lysate preparation, optimal dilutions for RSV-A2 antigen, the serum dilutions and generation of a standard curve from a pooled adult serum were determined by a checkerboard titration as previously described 21,22 . In every run, one half of the 96 well plate (column 1-6) was coated with 50µl/well of RSV lysate (antigen), while the other half (column 7-12) was coated with 50µl/well of mock lysate (mock). The mock consisted of Hep2 cells without RSV virus prepared using same procedure as that of the RSV lysate. Plates were incubated overnight at 37°C, then blocked for 1 hour with 200µl/well of 5% skimmed milk at 37°C. Blocking buffer was flicked off. Diluted serum samples 100µl/well were dispensed to both the antigen and mock sides of the plate. The plates were washed 4 times with 200µl/well of 0.05% Tween 20 in PBS (PBS-T) using an ELISA plate washer. A secondary antibody [polyclonal antibody to human IgG heavy chains (Goat anti human IgG HRP antibody-KPL, Catalogue# 074-1002) (100µl/well) diluted 1:1000 in PBS buffer was added to each well and incubated for 1 hour at room temperature. The reaction was developed using 50µl/well of Ortho-Phenylenediamine dihydrochloride (OPD, Catalogue# P8412-100TAB, Sigma-Aldrich) solution as substrate (prepared just before use in the ratio 1mg of OPD in 1 ml of PBS and 1ul of hydrogen peroxide). The intensity of colour developed was read at 490nm using an ELISA reader (SYNERGY 4, BioTeK). All samples were run in duplicate to monitor reproducibility of results and variability arising from pipetting. Positive and negative controls were run on every plate and plotted on a graph over time to check for antigen deterioration of the standards or coating antigen. The OD values of the mock were subtracted from the OD values of the antigen lysate to give the final OD value of the serum sample. OD values of the standard serum dilutions were assigned arbitrary unit (AU) values. Serum samples were assigned arbitrary units of RSV specific IgG by comparison against a standard curve generated from the pooled adult serum (serum standard) tested in each ELISA plate.

Commercial RSV IgG ELISA (IBL ELISA).
Use of the commercial RSV IgG ELISA kit and the procedure followed manufacturer's guidelines as described in the package insert (Cat#: RE56881-IBL International GmbH, D-22335 Hamburg, Germany). The ELISA procedure was based on a sandwich principle. In this assay, the 96 well plates were pre-coated with Vero E6 cells; strain long, F-protein; cell extract of RSV subgroup A (long strain) containing F-protein as well as G-protein as antigen (IBL communication). Specific antibodies from the serum samples binding to the pre-coated antigen were detected by a secondary antibody, rabbit anti-human conjugated to horse radish peroxidase enzyme. The reaction was developed using Tetramethylbenzidine(TMB) solution as substrate. The intensity of the colour developed, which is proportional to the concentration of IgG-specific antibodies detected, was read at 450nm using an ELISA reader (SYNERGY 4, BioTeK). The concentration of RSV IgG antibodies from the samples was estimated by comparison with a standard curve fit using a four parameter logistic regression model conducted in a GraphPad Prism software version 7.03.

Statistical analyses
Data analysis was conducted using STATA version 13.1 (College Station, Texas). The laboratory data were merged with archived data from the KBC participant' databases for analysis. Sample PRNT titres were logarithmically transformed (base 2) for all statistical analyses.
Correlation analyses between PRNT antibodies (log 2 PRNT) and IgG ELISA (log 10 ) antibodies was done using a Pearson correlation test. Agreement between PRNT and ELISA was done using Bland-Altman plots 23 . For each serum sample, the difference between PRNT and ELISA result was plotted against the averages of both assays. During Bland-Altman analysis, concentrations for RSV IgG IBL ELISA and crude lysate RSV IgG ELISA were log transformed to (base 2) and the data tested for normality. Modified Bland-Altman plots were used to test the effect of age on agreement between ELISA and PRNT. Regression analysis between ELISA and PRNT was used to estimate the 95% prediction intervals of PRNT antibodies by IgG ELISA method. The standard error (SE) of the predicted PRNT titre for an individual was estimated using the formula: Where, RMS is the residual mean squares and n is the number of observations. The predicted mean of PRNT titre by an ELISA method was calculated using the formula: , Where y is the predicted PRNT titre, m is the regression coefficient, x is the RSV antibody concentration by ELISA measurement and c is the y intercept where x=0. The upper and lower 95% prediction intervals were calculated using the formula: Upper/Lower 95% prediction limit = (mx+c) +/-1.96SE, where SE is the standard error (see full methods here).

Baseline characteristics
Archived sera from 100 cohort participants were selected. However, for 1 participant, the samples were not sufficient to screen for ELISA IgG antibodies after screening for PRNT titres. A total of 229 samples (99 cord samples, 87 three-month samples and 43 six-month samples) from the 99 participants were therefore available with data for the 3 serological assays. The mean gestational age at birth was 38.5 weeks (95% CI 37.6 -39.3), while the mean birth weight was 2.9 kilograms (95% CI 2.7 -3.0). The mean concentration of log 2 PRNT titres were: cord blood samples, 10.6 log 2 PRNT (95% Confidence interval, CI; 10.3-10.8); 3 months of age samples; 8.3 log 2 PRNT (95% CI 8.1 -8.6); and 6 months and over 7.6 log 2 PRNT (95% CI 7.1 -8.1). The highest PRNT titre observed among these participants was 12.9 log 2 PRNT while the lowest was 5.1 log 2 PRNT.

QA results
The mean PRNT titre results of a panel of 5 reference samples from BEI Resources tested in our laboratory were within range of the PRNT titre results as tested in BEI in 2011. These results are shown in Table 1.

Comparison of ELISA with PRNT
A Pearson correlation test to determine the strength of association between ELISA antibodies and neutralizing antibody (Figure 1), showed a strong positive correlation between PRNT titres and IBL ELISA antibody (r=0.75), and moderate positive correlation between PRNT titres and crude ELISA antibody (r=0.61). The two correlations were each statistically significant (P=0.0001).
The distribution of log transformed RSV antibody concentrations were approximately log-normal for each ELISA dataset and for the PRNT data. Bland-Altman plots were drawn to assess the agreement between PRNT and ELISA in quantifying a high or low RSV specific antibody level from a sample (Figure 2). In this analysis, 95% of samples were observed to fall within the 95% level of agreement between PRNT and IBL ELISA and 96% of samples between PRNT and crude ELISA, respectively. The Bland-Altman plots showed a mean difference of 4.5 log 2 PRNT (95% limits 1.3-7.6) between PRNT and IBL ELISA and 2.1 log 2 PRNT (95% limits -1.9-6.2) between PRNT and crude ELISA. Crude RSV lysate IgG ELISA showed a slightly wider 95% limit of agreement (-1.866, 6.157) compared to IBL RSV IgG ELISA (1.392, 7.595).
Modified Bland-Altman plots (Figure 3) show the difference in value of the ELISA and PRNT results plotted against age in months (i.e. time since birth of collection of the samples). Age did not have any appreciable effect on the agreement between PRNT and either of the ELISA result.
Further assessment using regression analysis (Figure 4), estimated the standard error for predicting the mean PRNT titres by ELISA     method to be 1.204 and 1.424 for IBL ELISA and crude ELISA, respectively. All samples with low concentrations of RSV specific antibody were within the 95% prediction limit for crude RSV lysate IgG ELISA (Figure 4b). The predicted PRNT values by IBL ELISA were given as (1.880x + 6.517), while the predicted PRNT values by crude ELISA were given as (1.419x +6.164). The 95% prediction limits for PRNT antibodies were slightly higher for crude ELISA +/-2.79 log 2 PRNT compared to IBL ELISA +/-2.36 log 2 PRNT.

Discussion
To overcome the challenges experienced with PRNT as a technique for quantifying protective immunity correlates to RSV in large vaccine studies, we explored how well ELISA and PRNT methods agree in detecting levels of RSV specific antibodies, and in addition, investigated how accurate an ELISA method can predict the PRNT measurements and if it could be considered a suitable replacement for PRNT.
We found moderate to good correlations between each ELISA (IBL (r =0.75) and crude (r=0.61)) and the reference method for neutralizing antibodies, PRNT. The stronger correlation in IBL ELISA could be explained by the fact that, both PRNT and IBL ELISA methods were designed to target specific antibodies against the RSV F and G protein. This is because, neutralising antibodies are presumed to dominate within the RSV F protein also commonly known as the key target for neutralizing antibodies 24,25 . Consequently, there is increasing interest to use the RSV F protein as the maternal RSV vaccine candidate antigen 7 . The moderate correlation with crude ELISA is likely caused by the wide range of antibodies against multiple RSV antigens the assay targets. This might have possibly led to detection of additional antibodies specific to other RSV proteins that are missed by both IBL ELISA and PRNT assay.
We identified few studies that show correlation between ELISA and neutralization assay [26][27][28][29] . Similar findings to our study were previously reported by Welch et al, who suggested that ELISA measure all antibody types and do not discriminate the neutralizing antibodies as measured by functional assays, thus cannot be relied upon to predict the neutralizing activity of the sera 29 . Due to the moderate correlation observed in our study between PRNT and crude RSV lysate IgG ELISA, we suggest that, careful consideration should be made on the choice of ELISA based assays as a surrogate for a neutralization assay in epidemiological studies.
The Bland-Altman plots demonstrated that 96% of all samples fell into the 95% limits of agreement between PRNT and crude ELISA. However, the crude ELISA had a slightly wider limit of agreement with PRNT compared to IBL ELISA. The PRNT values were also consistently higher than for ELISA. The wider limit of agreement indicates that the crude RSV lysate IgG ELISA has a higher variability with PRNT compared to IBL ELISA. This again, raises the issue of specificity and affinity of binding for RSV specific IgG antibodies measured using the crude ELISA method. Very few studies have used Bland-Altman plots to assess agreement between two serological methods 29-32 .
One study which evaluated two commercially available ELI-SAs and one in-house reference laboratory ELISA for the determination of human anti-rabies virus antibodies 29 , suggested that the results from the Bland-Altman plot analysis can only offer an insight into the availability of alternative, less complex method to monitor antibody titres during vaccine studies.
We also demonstrated that age did not have an appreciable effect on the agreement between PRNT and ELISA methods in detection of RSV specific antibodies. We performed this analysis because poor correlation between functional antibody and antibody concentration is thought to be influenced by age 33 . For instance, samples from infants and the elderly are likely to show a poor correlation between PRNT and ELISA usually because their receptors don't activate appropriate T or B cell responses, although the antibodies bind to antigen 33 . However, in this study, both methods showed agreement in the level of detected antibody for an individual regardless of age.
In this study, we could predict PRNT titres using ELISA with moderate accuracy. Using regression analysis, the IBL ELISA method predicted the mean PRNT titre at 95% prediction interval within a width of 4.8log 2 PRNT, while, crude RSV lysate IgG ELISA predicted the mean PRNT titre within a width of 5.6 log 2 PRNT units. This implies that, the PRNT titre of any given serum sample if estimated using an ELISA IgG antibody concentration measured directly by the ELISA method would fall within 2.4-2.8 fold higher or lower than the true value. In a perfect regression, if RSV antibody concentrations were measured by ELISA, the corresponding PRNT values would be estimated using (1.880x + 6.517) and (1.419x +6.164) for IBL ELISA and crude ELISA respectively. With, x being the antibody concentration measured directly by the respective ELISA method. However, the high variability makes direct estimation of PRNT values by the above formula difficult. For instance, the 95% upper limit of detecting mean cord PRNT titre of 10.6 log 2 PRNT by crude ELISA would be 13.4log 2 PRNT, which is above the highest value (12.9log 2 PRNT) the PRNT assay detected in this study. The combined analysis approach we have applied on the data is unique and we think, this could better guide the choice of a suitable serological technique for use in place of a PRNT assay for large vaccine studies; and especially if the units of measurements between the two methods are different.
The main limitation we highlight in this study is that the PRNT assay we are using is difficult to standardize, which complicates comparison of results between laboratories. However, during the time this study was conducted there was no RSV standard available. Nevertheless, we tried to mitigate this limitation by using a panel of reference serum samples from BEI resources to monitor and minimise assay variability and to make sure results of the test samples obtained are acceptable. Another limitation is the systematic bias observed with analysis using Bland-Altman plots. This was due to the differences in metrics used to measure RSV specific antibodies by ELISA and PRNT. The RSV specific antibodies in ELISA are measured as a concentration (log10) while in PRNT assay are measured as titres (log2). The Bland-Altman plots show a trend if analysis is done using the respective metrics of the serological assays, thus difficulty in estimating the mean difference. The plots in Bland-Altman analysis should demonstrate that the two methods are consistent in what they are measuring in the same metrics. We tried to minimise this by transforming ELISA values to log base 2 during Bland-Altman plot analysis.

Conclusions
Both ELISA methods show a moderate-good correlation with PRNT in measuring RSV specific antibodies. However, a commercial anti RSV based ELISA antibody assay which has less variability and has a high positive correlation with PRNT assay in detecting RSV specific antibody responses, does not provide an accurate prediction of PRNT antibody values. The true PRNT value would lie 2.4 fold higher or 2.4 fold lower if a commercial RSV IgG IBL ELISA is used in place of a PRNT assay. Our results suggest that, should PRNT prove unsuitable as a Gold standard for quantifying RSV specific antibodies, the utility of an ELISA method in vaccine studies should be assessed independent of a PRNT assay.

Data availability Underlying data
The raw data is stored under restricted access and available from the authors upon request through submission of a request form http://kemri-wellcome.org/aboutus/#ChildVerticalTab_15 for consideration by our Data Governance Committee (dgc@ kemri-wellcome.org). The major objective of this manuscript was to determine if at least one of the two in house RSV ELISA assays could predict with accuracy the neutralizing antibody titers generated by a plaque reduction neutralization test (PRNT). Although the methodology used to make the comparisons and analytical approach used to make the prediction are sound, the basic assumption is flawed when using crude or poorly described viral antigens in the ELISA assays. The neutralization assay primarily measures neutralizing antibodies directed to site-specific sites on the F protein while the ELISA assay using viral antigens will measure binding antibodies direct to the surface glycoproteins (F, G & SH) as well as the internal proteins (N, P, M, M2). Although good correlations are normally seen between ELISA and neutralization assays, it is entirely a different proposition to predict with accuracy neutralizing antibody titers when using viral antigens in an ELISA assay. Even site-specific competitive antibody assays to the F protein are not able to predict with accuracy neutralizing antibody tiers measured by neutralization assays because the immune response of individuals vary to the antigenic sites (Ø, II, IV, III, V) of the F protein even though they may have comparable neutralizing antibody activity. The lack of precision was evident in the results with the potential to have up to a 16 to 32-fold swing in the neutralizing antibody titer prediction using the ELISA assays. .

Other comments
In the manuscript PRNT is described as the preferred technique to determine immune correlates of protection. It is important to point out that the PRNT is one of many neutralization assays that have been developed for measuring neutralizing antibody activity against RSV. There are two major classes of neutralization assays; the PRNT and the microneutralization assay. There are many assay variations depending on the cell line, the virus inoculum, the signal or read out used to measure neutralizing activity, the duration of the assay, etc. Many of the PRNT references used in the manuscript actually refer to the classical microneutralization assay. It is also important to note that these neutralization antibody assays, in particular those without complement, primarily detect neutralizing antibody directed to the F protein and not the G protein.
Neutralization assays are considered the best current method to evaluate the protective potential (neutralizing antibodies) induced by RSV infection or candidate vaccines. We now have an international standard (National Institute of Biological Standards and Control-NIBSC) against RSV/A (also being developed for RSV/B) that can be used to adjust neutralizing antibody titers generated by different neutralization assays into IU. This will greatly facilitate the comparisons of neutralizing antibody activity measured by different assays and induced by different candidate vaccines. The introduction section should be revised to reflect these up-dates. measured by different assays and induced by different candidate vaccines. The introduction section should be revised to reflect these up-dates. .

Introduction
The Fusion (F) protein in Novavax F nanoparticle vaccine is not a true post-fusion F protein vaccine. It is consistent with a prefusogenic form that is able to generate site-specific antibody responses to both the pre-fusion and post-fusion F conformations . .

Method
What was the collection tube used to process plasma? Plasma collected in heparin coated tubes will inaccurately measure plasma samples with low neutralizing antibody activity to RSV because heparin containing plasma will act as an antiviral agent. The effect is to inhibit virus replication when in reality there is no neutralizing antibody activity. This is unlikely to have an effect with plasma samples with moderate to high levels of neutralizing antibody titers against because the effect is diluted out with increasing two-fold dilutions. For the ELISA assays it is unclear what the dilution scheme was for generating the ELISA units. The results are presented in log10, which suggest a dilution scheme of 10-fold. Please clarify the dilution scheme for the two ELISA assays. Also for all the assays please describe the range of the antibody titer and arbitrary unit for the assays. It would also be helpful to know the coefficient of variation for these assays at the lower limit of detection, dynamic range and upper limit of detection. It is unclear the viral antigen(s) included in the IBL ELISA assay. The information is not provided in the IBL package insert. .

Results
It is a misnomer to call it an RSV-F ELISA. Please see Y axis for Figures 2 and 3. Figures 2 & 3, Y axis is the difference (PRNT -RSV F protein ELISA) in log ? What was the correlation between the two ELISA assays, and did one ELISA assay predict with good accuracy the AU of the other ELISA assay. That would serve as an internal validation of the statistical method used to predict neutralizing antibody titer with ELISA bind antibody data. .

Discussion
Page 8, 7 paragraph, "predict with moderate accuracy". Having a width of 4.8 log to 5.6 log is huge and should be considered imprecise. In my opinion, the main limitation is not the standardization of the PRNT but rather the assumption that accurate predictions can be made when using ELISA assays with poorly characterized viral antigens. A second major limitation is the use of immunological assays that do not appear to have been qualified. Information on the reproducibility, dynamic range, lower and upper limit of detection were not provided for these assays. lysate, which as the authors point out, is likely due to targeting of F and G antigen in the commercial assay. Despite this higher correlation, the margin of error for the formula calculating the relationship between the ELISA and the PRNT is too high to be useful. This is a useful study, and reaffirms the use of RSV neutralizing antibody assays as the gold standard. One minor criticism, largely outside the scope of this study, is the consideration of other correlates of protection beyond neutralizing antibody as markers of protection against disease.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound? Yes

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Yes This study evaluated the agreement between the RSV plaque reduction neutralisation assay (PRNT), a commercially available RSV IgG ELISA assay and an in-house RSV assay using crude RSV lysate as an antigen. General comments: Overall, the methods and results section of the manuscript are well written and analysed. Although the study objective is clear, the rational for the study is more ambiguous. The main issue at hand in the context of vaccine development, is the need for an assay which can predict the efficacy of the vaccine -and ideally a functional assay or at least one that bridges with a functional assay. This would be important for bridging studies, in the absence of being able to undertake multiple large scale vaccine efficacy/effectiveness studies. Although the authors cite some studies reporting a positive association of PRNT antibody and risk/odds of disease, omitted from the 2.
reporting a positive association of PRNT antibody and risk/odds of disease, omitted from the background are other studies (including one from the same group)-which actually reported no association between PRNT and RSV LRTI. This calls into question the value of the current "gold standard"-i..e PRNT-on its own (over and above the issues with regard to standardisation and being labour intensive) in relation to whether or not it would be useful for establishing a correlate of protection.
In the context of this study where the ELISA assays, as also noted by the authors, are not discriminatory in terms of measure of antibody to epitopes specifically targeted by most of the vaccines currently under clinical development (including the nanoparticle F-fusion vaccine), the likelihood of any predictive value of a relatively non-specific ELISA assay being of use to establish or impute a correlate of protection would be doubtful from the outset. These issue merit inclusion in the abstract, study rational and the discussion. The measure of RSV antibody could have multiple purposes-including characterising sero-epidemiology, measuring vaccine immunogenicity or trying to establish a correlate of protection.
Depending on for what purpose the assay is being used, would influence the minimum characteristics required of the assay. In the context of this study, although the ELISA assays were only moderately predictive of the PRNT antibody measure, this does not necessarily preclude the use thereof in sero-epidemiology studies. Likewise, even if there was a strong correlation between the ELISA and PRNT assays, in the absence of an established correlate of protection using the PRNT assay-that too would not be of much value in terms of being used for studies to impute vaccine effectiveness in different settings.

Specific comments
Pg 3 end para 2-need to clarify that the standardised assays which are required need to be assays that are predictive of vaccine protection. Also, clarify that the main purpose of such assays are to undertake "bridging immunogenicity studies" to avoid multiple vaccine efficacy/effectiveness studies. Such assays will need to be tested for predicting vaccine efficacy often within a clinical trial and the assays could differ based on the construct of the vaccine (i.e differences in specific epitopes targeted). Pg 3 para 3-mention and reference studies that have shown no association between PRNT and LRTI. Pg 3 para 4-clarify that the needs of an assay for sero-epi studies may well differ to those required to establish a correlate of protection. Pg 3 para 5-what would the need for a serological assay be in a "large vaccine efficacy trial" which will provide the most definitive answer on vaccine efficacy? Under lab methods-clarify where the lab assays were done. Pg 8 para 3-neutralizing antibodies are also elicited by immune responses to the G and M protein.
This might well explain some of the conflicting findings on the association of PRNT and risk for RSV LRTI. Pg 8 para 4-the last sentence needs to be elaborated upon, including what the possibility for epitope specific antibody read-outs that might well differ between vaccine constructs.

Is the study design appropriate and is the work technically sound? Yes
Are sufficient details of methods and analysis provided to allow replication by others? Yes