Comparing antibody neutralization activity of different sera against genetically and antigenically diverse viral strains requires standardization. ID50 (or ID80) values, the inhibitory dilutions at which 50% (or 80%) neutralization is attained, are determined for a panel of viruses, using the TZM-bl neutralization assay (Sarzotti-Kelsoe et al., 2014). Serum breadth and potency are two measures used to characterize neutralization responses across virus diversity. Breadth is the proportion of pseudoviruses with an ID50 score above the threshold of detection, and potency is the geometric mean ID50 (Hraber et al., 2014; Rademeyer et al., 2016). At least half of the variation in neutralization assay results from large panels can be explained by the averaged responses per serum, Env, and the entire panel, overall (Hraber et al., 2014). Serum breadth and potency therefore depend strongly on the Env panels used, which can vary markedly between studies.

Virus neutralization sensitivity to panels of sera from chronically infected individuals represents a continuum (Seaman et al., 2010). To characterize Envs in tiers involves partitioning large neutralization panels into three or four groups with similar sensitivity (Rademeyer et al., 2016; Seaman et al., 2010). Antibodies able to neutralize only tier 1 (most sensitive) viruses are readily elicited by HIV Env gp120 immunogens, but such tier1 responses are not protective; in human vaccine efficacy trials, such responses have been unable to confer protection against the viruses that continue to fuel the pandemic (Gilbert et al., 2010; Montefiori et al., 2012). Tier 2 viruses are more difficult to neutralize than tier 1, and represent the majority of viruses that are transmitted to establish new infections (Rademeyer et al., 2016; Seaman et al., 2010). Tier 3 viruses are the most resistant to neutralization.

One difficulty with the tiered scheme for labeling viruses (i.e. tiers 1A, 1B, 2, and 3) is that it simplifies a continuous distribution into three or four categories (Seaman et al., 2010), despite wide variation within each category. Moreover, while the system categorizes viruses, it does not help compare serum neutralization potency. For example, a serum that neutralizes one tier 3 virus but only a few tier 2 viruses might subjectively be designated a ‘tier 3 neutralizing serum,’ while one which neutralizes no tier 3 viruses but many tier 2 viruses a ‘tier 2 serum.’ The latter serum is likely more potent (protective) in real-world scenarios despite being designated with a lower tier. A metric to rate sera for neutralization potency would be useful, for example to down-select vaccine candidates for further evaluation in clinical trials. Such a metric should be objective and continuous, rather than category-based. It should also provide biologically meaningful and interpretable values that are consistent with expectations of tiered viruses from terminology used by practitioners in this field.

Here, we describe an objective, quantitative metric for serum classification, and apply it to characterize serum neutralization activity against both large and smaller panels of pseudoviruses. It uses logistic regression to establish a numerical value for a given serum, based on its ability to neutralize viruses of different tiers. We describe a statistically motivated Neutralization Potency (NP) score, which represents serum neutralization tier on a continuous, rather than categorical, scale. That scale is designed to be intuitively meaningful to HIV researchers, such that sera with a low score (near 1) are able to neutralize only tier1 viruses, while sera with scores ranging from 2 to 3 reflect increasing capacity to neutralize tier 2 and 3 viruses. A continuum of NP values enables comparisons between sera. Rather than suggesting most sera can neutralize tier 2 viruses, NP values can distinguish between, say, ‘tier 2.1’ and ‘tier 2.5’ sera, the higher score indicating a better neutralization outcome. The potency comparisons are similar to comparing geometric mean neutralization titers, but instead are represented in tier-like terms.

Because this approach is based on the outcome of yes-or-no neutralization evaluations from a single dilution of serum, it can be used to evaluate large numbers of sera in novel high-throughput designs. The examples here use a threshold ID50 of 1:50, that is a binary assignment of whether or not a serum neutralizes 50% of a virus at a dilution of 1:50. This means the NP could be calculated from a single serum dilution, as opposed to a full eight-point titration series. The metric lends itself to high-throughput methods to compare neutralization potencies of many sera.

To define a metric that can compare neutralization potencies of different sera, we assigned a single neutralization index (NI) value to each HIV-1 envelope-pseudotyped virus (Materials and methods). The logarithm of the geometric mean ID50 against 205 sera was linearly rescaled to correspond with tier designations. The resulting NI values thus capture envelope neutralization sensitivity across 205 sera and provide a continuous scale of sensitivity that roughly corresponds to their tier designations. For logistic regression analyses below, the Envs were evaluated according to their NI. We note that the neutralization index could be defined in other ways, for example by using area-under-curve (AUC) values from the dilution series (Yu et al., 2012).

Tier-scaled virus NIs can be used to quantify serum neutralization activity. As higher-tier viruses are more difficult to neutralize, the expectation for a typical serum is that it can more potently neutralize lower tier viruses than higher tier viruses. In an over-simplification to illustrate the concept, we consider cases where a single threshold NI value cleanly separates outcomes, such that only those viruses above that NI resist neutralization by that serum. The serum would be assigned the threshold NI as a measure of neutralization potency. For example, one hypothetical serum that neutralizes tier 1 viruses up to tier 1.1 would receive a neutralization score of 1.1 (Figure 1a). Another hypothetical serum that neutralizes tier 2 viruses up to tier 2.8, receives a score of 2.8 (Figure 1b). Thus, in these simple examples, the scores for sera directly reflect their ability to neutralize viruses up to a rescaled virus tier value.

Figure 1

Download asset Open asset

In practice, neutralization responses are noisy and not clearly resolved as in these two examples. Instead, the viruses neutralized by a particular serum are more scattered, and the overall trend becomes apparent in the aggregate view across many viruses, by considering how the probability of neutralization depends on rescaled tier values (Figure 1c). The probability for a serum to neutralize should decrease as virus NI values increase. We define serum neutralization potency (NP) as the NI that gives equal probability for viruses sensitive and resistant to neutralization by that serum, which we estimate using logistic regression (Figure 1d and Materials and methods).

Serum neutralization potencies from 225 Envs

From the large, multi-clade, M group neutralization panel (Hraber et al., 2014), we computed log-geometric mean neutralization ID50 titers per virus and serum, then transformed the log-means onto the interval from at least 1 to below 4, to obtain tiered neutralization indices (NIs). This transformation gives an inverse relationship between tier-scaled NIs and geometric mean titer for viruses, as higher titers correspond to lower-tier viruses (Figure 2a).

Figure 2 with 1 supplement see all

Download asset Open asset

Conversely for sera, higher titers averaged across viruses indicate greater probability of neutralizing higher-tier, neutralization-resistant viruses (Figure 2b). That is, tier 3 sera (NP >3) are better able to neutralize tier 3 viruses than are tier 1 sera (NP <2). A tier 2.5 serum should generally be able to neutralize viruses up to that point on the neutralization sensitivity continuum, although there could be some viruses that are neutralized above and a few resistant below. Low scatter, as illustrated in Figure 1a and b, gives a steep slope in logistic regression, indicating a sharp boundary between viruses neutralized and not neutralized. More scatter, that is lower contrast between viruses neutralized and not neutralized with increasing mean ID50, as illustrated in Figure 1c, gives a lower slope. Inability to resolve between neutralization outcomes would give no slope, quantified by a high probability of a false positive (p value) from rejecting the null hypothesis that the slope is zero, as is true for some NP outcomes in Figure 2b. This is most common at the low end of the serum neutralization continuum, where the ability to neutralize a virus is constant (and low) across the range of virus sensitivities.

Resampling (with replacement) sets of 225 Envs from the M group panel indicates that NP values are robust to sampling variation, save for a few sera with a slope of zero (Figure 2—figure supplement 1a). Variation may increase slightly among resampled NP values at the extremes of the neutralization scale even when the slope is non-zero (Figure 2—figure supplement 1b).

Neutralization responses for a typical serum, such as SA.C37, which has median potency among the sera we studied, appear in Figure 3. For each virus, the neutralization outcome is shown as a function of the tier-transformed geometric mean ID50 (Figure 3a). Serum neutralization potency computed from the 225 Env-panel is 2.5. Separation, with overlap, between viruses neutralized and not neutralized is apparent in Figure 3b.

Figure 3

Download asset Open asset

Serum neutralization potencies from small Env panels

As described in Materials and methods, we identified smaller panels and evaluated whether sets of 11 of the 12-Env global panel (deCamp et al., 2014), or either 10 or 20 Envs identified by lasso (Tibshirani, 1996; Friedman et al., 2010) could estimate neutralization indices more efficiently than the full set of 225 Envs. Together, five Envs were shared by all three small panels (Figure 4—figure supplement 1) and four additional Envs were common to both the global panel and 20-Env lasso panel, while the two hand-selected Envs were specific only to the global panel. The Envs that were selected to infer NP from smaller panels represent a range of neutralization sensitivities, favoring more sensitive and disfavoring the insensitive viruses (Figure 4—figure supplement 1).

A review of the ability of each panel to model NP for the serum SA.C37 (Figure 4) suggests that the 20-Env panel, as might be expected, is better able to resolve between neutralization outcomes (p=0.000162, Figure 4c) than smaller panels (Figure 4a and b; p=0.00298 and p=0.00151 for global and 10-Env panels, respectively). In this example, the standard global panel performed as well as the lasso panel of 10. This one case may not indicate the responses when tested across more sera.

Figure 4 with 3 supplements see all

Download asset Open asset

NP values inferred from smaller panels are correlated with NP values from larger sets of holdout, non-panel Envs, and also with each other, across panels (Figure 4—figure supplement 2). The lasso-selected panel of 10 and the global panel perform similarly, so the global panel performs reasonably well for a panel of that size. The global panel may offer greater resolution for tier 1A and 1B sera, because it includes more Envs with NI below 2.0 than either of the panels selected by lasso. With panel Envs, inferred serum NP values were roughly limited to range from 1 to 4. When more Envs are tested, the NP values can fall below 1 or above 4, most likely because the most extreme Envs on the neutralization continuum allow the logistic regression to fit parameters outside the range typically expected.

Over all the 205 sera, the 20-Env panel is better powered to detect a non-zero slope than the smaller 10- and 11-Env panels (Figure 4—figure supplement 3), giving p>0.05 in about half as many cases as the smaller panels, likely because greater statistical power results from having nearly twice as many measurements to compute logistic regression parameters. Overall, rather than recommending use of a single panel to compute NP, it seems NP can be computed using a reasonable choice of Envs that represent a range of neutralization sensitivities, and use of more Envs is better able to quantify NP significantly than fewer Envs.

Neutralization responses in progressors and long-term non-progressors

Using previously reported neutralization assay results (Doria-Rose et al., 2010), we computed geometric mean ID50 titers from 20 Envs and 103 donor sera, of which 25 were previously found to be long-term non-progressors (LTNP) (Migueles et al., 2002; Migueles et al., 2008; Doria-Rose et al., 2009). We identified 10 Envs in this panel that were also tested in the M group panel, two from subtype C (DU422.1, DU156.12), seven from subtype B (BG1168.1, 6101.10, TRJO4551.58, PVO.4, CAAN5342.A2, THRO4156.18, TRO.11), and one from subtype A (Q769.D22). We computed NP values from these 10 Envs, using a cutoff ID50 of 50 for positive neutralization outcome, and p-value cutoff from χ² testing of 0.1 to indicate a significant NP score. This cutoff excluded 4 of 25 LTNP sera and 20 of 78 progressors; the proportions of excluded NP values were not significantly different between progressors and non-progressors (Fisher’s exact test, p=0.42). NP values are highly correlated with geometric mean ID50s (Figure 5), regardless of whether or not NP outcomes with high p-values from χ² testing are excluded (Kendall’s τ, p<2.2 × 10⁻¹⁶).

Figure 5

Download asset Open asset

While the range of NP values in Figure 5 may seem large, a score of 4.1 indicates sera that neutralized all ten Envs (ID50 ≥50), and the lowest-scoring sera neutralized none. To help interpret this range, consider the serum with a nominal NP of 4.6, which had a high corresponding p-value of 0.24. This particular serum neutralized 9 of 10 Envs, but the NP score is not significant. The only Env this serum failed to neutralize was DU156.12, which should be the most readily neutralized of these 10 Envs. (Its NI is 2.04, versus a mean NI of 2.91 and range from 2.46 to 3.34 among the other nine.) In this case, DU156.12 may contain one or more mutations that altered an epitope targeted by this serum.

As anticipated, based on the established studies (Doria-Rose et al., 2010), the geometric mean ID50s differed significantly between the progressors and non-progressors (Wilcoxon test, p=2.1 × 10⁻¹¹). We noted the same outcome for breadth, defined as the percentage of 20 Envs neutralized per serum with an ID50 of at least 50 (Wilcoxon p=7.2 × 10⁻¹¹). Similarly, NP values differed significantly between these groups (n = 79, Wilcoxon p=4.3 × 10⁻⁹). This NP comparison therefore agrees with established findings (Doria-Rose et al., 2010), but notably, the NP comparisons involved half as many Envs and could use many fewer dilutions than required to compute mean ID50s among 20 Envs. To repeat the NP analysis using single-dilution assays, rather than a five-point (or greater) dilution series, the entire comparison would require at most one-tenth the number of neutralization reactions and material per sample, compared with the experiment using mean ID50 titers. Also, as seen above, testing more than 10 Envs per serum could increase statistical power, and reduce the number of sera excluded because their NP scores were deemed insufficiently significant.

Antibody combinations

The same methods can be used with data from titration experiments using monoclonal antibodies. Although the scale of measurement is reversed (low IC50 values indicate high potency), a cutoff value can again be used to obtain a yes-or-no neutralization response. Using data from an earlier study (Kong et al., 2015), we explored the behavior of NP values from broadly neutralizing antibodies (bnAbs), both alone (monoclonal) and in combinations of up to four bnAbs. The NP and also the slope of the logistic function increase as bnAbs are combined in most cases, but not all (Figure 6). These results suggest that, all else equal, sera with higher number of antibody specificities could have higher NP and slope values.

Figure 6

Download asset Open asset

We have described a simple method to quantify and compare serum neutralization probabilities. The method uses logistic regression to model the probability that a serum neutralizes a virus with an ID50 titer above some cutoff. The neutralization potency (NP) identifies where the probabilities of neutralizing and not neutralizing a virus are equal. It provides a continuous measure for sera, which builds upon established tier categories now used to rate virus sensitivity. The NP statistic defines the greatest virus tier that a serum can be expected to neutralize. Thus, an NP of (roughly) 1.8 to 2.8 is ‘tier 2-like’, and an NP above 2.8 is ‘tier 3-like’. NP values below 1 are unable to neutralize even tier 1A Envs. The NP values are not absolute and depend on the ID50 cutoff used.

Defining a neutralization potency by testing a serum against a panel of 225 Envs on a routine basis is impractical and costly. We identified subsets of these Envs that largely reproduce the results from testing all 225. This makes assignment of NP values to a set of sera far more tractable than testing against all Envs. The already established 12-virus global panel may suffice to characterize NP values, although larger panels tend to give more significant outcomes.

Evaluating neutralization assay outcomes against the continuum of neutralization sensitivity among viral variants provides more context to interpret results, because it considers not only the proportion of tier 2 Envs neutralized (breadth), but which Envs should most likely be neutralized. This helps to interpret differences between sera that neutralize the same number of Envs, each of which have different sensitivities. It also helps to compare sera where titers may be averaged over many outcomes below the limit of assay quantification, as was the case for sera from vaccinated rabbits (Table 2) (Torrents de la Peña et al., 2017).

We also explored results from experiments that utilized different Env panels and found they can be compared on the same neutralization scale (not shown). The ability to do this requires only that some number of Envs in each panel have available tiered neutralization scores, computed from available data. Better comparative results are obtained when using more Envs, because resulting NP values are less likely to be undefined.

A web-based utility – http://hiv.lanl.gov/content/sequence/NI/ni.html – at the Los Alamos HIV database computes NPs for sera tested against subsets of M group Envs. In addition to the analysis described here, it can also compute and report NPs for clade C Envs (Rademeyer et al., 2016) and for antibody IC50s.

Broadly neutralizing monoclonal antibodies are similarly characterized when isolated, and TZM-bl assay inhibitory concentration IC50 and IC80 scores are generally determined across large pseudovirus panels, for example, to characterize a newly isolated antibody (Wu et al., 2010). We applied the same analytic methods to IC50s from antibodies. Our analysis of data from experiments that combined broadly neutralizing antibodies (bnAbs) having distinct specificities suggests that the NP increases with the number or variety of distinct antibody specificities in the sample.

The NP, as defined here, is a single metric to compare serum potencies. However, logistic regression provides another parameter, the slope. The slope indicates agreement between serum neutralization outcomes and average potency (geometric mean ID50) among serum samples used to compute the NI per Env. We hypothesize that the slope should be low for sera with limited epitope breadth. In the extreme case of a serum that targets a single epitope (e.g. a monoclonal response dominates), only Envs with that epitope would be neutralized, and neutralization outcomes should be widely scattered among viruses tested, independent of an overall sensitivity of the virus to neutralization, resulting in a slope near zero. Thus, the slope may indicate diversity of epitopes targeted by the test serum. Consistent with this, we found single monoclonal bnAbs had lower slopes than mixtures of monoclonal bnAbs. Such a finding might help characterize the mixtures of antibody specificities in polyclonal sera, to complement the methods for computational neutralization fingerprinting that have recently been advanced (Georgiev et al., 2013; Doria-Rose et al., 2017). To evaluate this idea, subsequent work could identify serum samples with similar NP values but different slopes and map the epitopes therein.

In addition to establishing a metric for serum neutralization, a primary advantage of this approach is that it suggests a strategy to simplify neutralization assay experiments, for more cost-effective screening of responses in large-scale vaccination studies. Logistic regression utilizes a collection of binary (‘true’ or ‘false’) responses, rather than the actual neutralization titers. Because its derivation relies only on a single-dilution analysis, rather than the current eight-point dilution series, it can enable a larger-scale screening throughput. Sera that score well using this screening metric could be prioritized for a more thorough dilution-series analysis. Thus, our approach provides a relatively simple metric by which serum neutralization of HIV viruses can be compared with a quantitative method. Together with the slope (which may indicate breadth of epitopes targeted), this approach could be useful to down-select vaccine candidates and move forward with regimens that are able to elicit, on average, greater NP values.

In summary, we propose a way to simplify the comparison of neutralization potency of antisera. The resulting metric is continuous, scaled to provide an easily interpreted value, and may provide a formal method for ranking antisera. It is used on single dilution assay data, lending it to a high-throughput platform.

1. 1. Crawley MJ

   (2002)

   Statistical Computing: An Introduction to Data Analysis Using S-Plus

   513–536, Proportion data: binomial errors, Statistical Computing: An Introduction to Data Analysis Using S-Plus, Hoboken, Wiley & Sons.

   • Google Scholar
2. 1. de Taeye SW
   2. Ozorowski G
   3. Torrents de la Peña A
   4. Guttman M
   5. Julien JP
   6. van den Kerkhof TL
   7. Burger JA
   8. Pritchard LK
   9. Pugach P
   10. Yasmeen A
   11. Crampton J
   12. Hu J
   13. Bontjer I
   14. Torres JL
   15. Arendt H
   16. DeStefano J
   17. Koff WC
   18. Schuitemaker H
   19. Eggink D
   20. Berkhout B
   21. Dean H
   22. LaBranche C
   23. Crotty S
   24. Crispin M
   25. Montefiori DC
   26. Klasse PJ
   27. Lee KK
   28. Moore JP
   29. Wilson IA
   30. Ward AB
   31. Sanders RW

   (2015) Immunogenicity of stabilized HIV-1 envelope trimers with reduced exposure of non-neutralizing epitopes

   Cell 163:1702–1715.

   https://doi.org/10.1016/j.cell.2015.11.056

   • PubMed
   • Google Scholar
3. 1. deCamp A
   2. Hraber P
   3. Bailer RT
   4. Seaman MS
   5. Ochsenbauer C
   6. Kappes J
   7. Gottardo R
   8. Edlefsen P
   9. Self S
   10. Tang H
   11. Greene K
   12. Gao H
   13. Daniell X
   14. Sarzotti-Kelsoe M
   15. Gorny MK
   16. Zolla-Pazner S
   17. LaBranche CC
   18. Mascola JR
   19. Korber BT
   20. Montefiori DC

   (2014) Global panel of HIV-1 Env reference strains for standardized assessments of vaccine-elicited neutralizing antibodies

   Journal of Virology 88:2489–2507.

   https://doi.org/10.1128/JVI.02853-13

   • PubMed
   • Google Scholar
4. 1. Doria-Rose NA
   2. Altae-Tran HR
   3. Roark RS
   4. Schmidt SD
   5. Sutton MS
   6. Louder MK
   7. Chuang GY
   8. Bailer RT
   9. Cortez V
   10. Kong R
   11. McKee K
   12. O'Dell S
   13. Wang F
   14. Abdool Karim SS
   15. Binley JM
   16. Connors M
   17. Haynes BF
   18. Martin MA
   19. Montefiori DC
   20. Morris L
   21. Overbaugh J
   22. Kwong PD
   23. Mascola JR
   24. Georgiev IS

   (2017) Mapping polyclonal HIV-1 antibody responses via next-generation neutralization fingerprinting

   PLOS Pathogens 13:e1006148.

   https://doi.org/10.1371/journal.ppat.1006148

   • PubMed
   • Google Scholar
5. 1. Doria-Rose NA
   2. Klein RM
   3. Daniels MG
   4. O'Dell S
   5. Nason M
   6. Lapedes A
   7. Bhattacharya T
   8. Migueles SA
   9. Wyatt RT
   10. Korber BT
   11. Mascola JR
   12. Connors M

   (2010) Breadth of human immunodeficiency virus-specific neutralizing activity in sera: clustering analysis and association with clinical variables

   Journal of Virology 84:1631–1636.

   https://doi.org/10.1128/JVI.01482-09

   • PubMed
   • Google Scholar
6. 1. Doria-Rose NA
   2. Klein RM
   3. Manion MM
   4. O'Dell S
   5. Phogat A
   6. Chakrabarti B
   7. Hallahan CW
   8. Migueles SA
   9. Wrammert J
   10. Ahmed R
   11. Nason M
   12. Wyatt RT
   13. Mascola JR
   14. Connors M

   (2009) Frequency and phenotype of human immunodeficiency virus envelope-specific B cells from patients with broadly cross-neutralizing antibodies

   Journal of Virology 83:188–199.

   https://doi.org/10.1128/JVI.01583-08

   • PubMed
   • Google Scholar
7. 1. Friedman J
   2. Hastie T
   3. Tibshirani R

   (2010) Regularization paths for generalized linear models via coordinate descent

   Journal of Statistical Software 33:1–22.

   https://doi.org/10.18637/jss.v033.i01

   • PubMed
   • Google Scholar
8. 1. Georgiev IS
   2. Doria-Rose NA
   3. Zhou T
   4. Kwon YD
   5. Staupe RP
   6. Moquin S
   7. Chuang GY
   8. Louder MK
   9. Schmidt SD
   10. Altae-Tran HR
   11. Bailer RT
   12. McKee K
   13. Nason M
   14. O'Dell S
   15. Ofek G
   16. Pancera M
   17. Srivatsan S
   18. Shapiro L
   19. Connors M
   20. Migueles SA
   21. Morris L
   22. Nishimura Y
   23. Martin MA
   24. Mascola JR
   25. Kwong PD

   (2013) Delineating antibody recognition in polyclonal sera from patterns of HIV-1 isolate neutralization

   Science 340:751–756.

   https://doi.org/10.1126/science.1233989

   • PubMed
   • Google Scholar
9. 1. Gilbert P
   2. Wang M
   3. Wrin T
   4. Petropoulos C
   5. Gurwith M
   6. Sinangil F
   7. D'Souza P
   8. Rodriguez-Chavez IR
   9. DeCamp A
   10. Giganti M
   11. Berman PW
   12. Self SG
   13. Montefiori DC

   (2010) Magnitude and breadth of a nonprotective neutralizing antibody response in an efficacy trial of a candidate HIV-1 gp120 vaccine

   The Journal of Infectious Diseases 202:595–605.

   https://doi.org/10.1086/654816

   • PubMed
   • Google Scholar
10. 1. Hraber P
    2. Korber BT
    3. Lapedes AS
    4. Bailer RT
    5. Seaman MS
    6. Gao H
    7. Greene KM
    8. McCutchan F
    9. Williamson C
    10. Kim JH
    11. Tovanabutra S
    12. Hahn BH
    13. Swanstrom R
    14. Thomson MM
    15. Gao F
    16. Harris L
    17. Giorgi E
    18. Hengartner N
    19. Bhattacharya T
    20. Mascola JR
    21. Montefiori DC

    (2014) Impact of clade, geography, and age of the epidemic on HIV-1 neutralization by antibodies

    Journal of Virology 88:12623–12643.

    https://doi.org/10.1128/JVI.01705-14

    • PubMed
    • Google Scholar
11. 1. Julien JP
    2. Lee JH
    3. Ozorowski G
    4. Hua Y
    5. Torrents de la Peña A
    6. de Taeye SW
    7. Nieusma T
    8. Cupo A
    9. Yasmeen A
    10. Golabek M
    11. Pugach P
    12. Klasse PJ
    13. Moore JP
    14. Sanders RW
    15. Ward AB
    16. Wilson IA

    (2015) Design and structure of two HIV-1 clade C SOSIP.664 trimers that increase the arsenal of native-like Env immunogens

    PNAS 112:11947–11952.

    https://doi.org/10.1073/pnas.1507793112

    • PubMed
    • Google Scholar
12. 1. Kong R
    2. Louder MK
    3. Wagh K
    4. Bailer RT
    5. deCamp A
    6. Greene K
    7. Gao H
    8. Taft JD
    9. Gazumyan A
    10. Liu C
    11. Nussenzweig MC
    12. Korber B
    13. Montefiori DC
    14. Mascola JR

    (2015) Improving neutralization potency and breadth by combining broadly reactive HIV-1 antibodies targeting major neutralization epitopes

    Journal of Virology 89:2659–2671.

    https://doi.org/10.1128/JVI.03136-14

    • PubMed
    • Google Scholar
13. 1. Marschner IC

    (2011)

    glm2: fitting generalized linear models with convergence problems

    The R Journal 3:12–15.

    • Google Scholar
14. 1. Migueles SA
    2. Laborico AC
    3. Shupert WL
    4. Sabbaghian MS
    5. Rabin R
    6. Hallahan CW
    7. Van Baarle D
    8. Kostense S
    9. Miedema F
    10. McLaughlin M
    11. Ehler L
    12. Metcalf J
    13. Liu S
    14. Connors M

    (2002) HIV-specific CD8+ T cell proliferation is coupled to perforin expression and is maintained in nonprogressors

    Nature Immunology 3:1061–1068.

    https://doi.org/10.1038/ni845

    • PubMed
    • Google Scholar
15. 1. Migueles SA
    2. Osborne CM
    3. Royce C
    4. Compton AA
    5. Joshi RP
    6. Weeks KA
    7. Rood JE
    8. Berkley AM
    9. Sacha JB
    10. Cogliano-Shutta NA
    11. Lloyd M
    12. Roby G
    13. Kwan R
    14. McLaughlin M
    15. Stallings S
    16. Rehm C
    17. O'Shea MA
    18. Mican J
    19. Packard BZ
    20. Komoriya A
    21. Palmer S
    22. Wiegand AP
    23. Maldarelli F
    24. Coffin JM
    25. Mellors JW
    26. Hallahan CW
    27. Follman DA
    28. Connors M

    (2008) Lytic granule loading of CD8+ T cells is required for HIV-infected cell elimination associated with immune control

    Immunity 29:1009–1021.

    https://doi.org/10.1016/j.immuni.2008.10.010

    • PubMed
    • Google Scholar
16. 1. Montefiori DC
    2. Karnasuta C
    3. Huang Y
    4. Ahmed H
    5. Gilbert P
    6. de Souza MS
    7. McLinden R
    8. Tovanabutra S
    9. Laurence-Chenine A
    10. Sanders-Buell E
    11. Moody MA
    12. Bonsignori M
    13. Ochsenbauer C
    14. Kappes J
    15. Tang H
    16. Greene K
    17. Gao H
    18. LaBranche CC
    19. Andrews C
    20. Polonis VR
    21. Rerks-Ngarm S
    22. Pitisuttithum P
    23. Nitayaphan S
    24. Kaewkungwal J
    25. Self SG
    26. Berman PW
    27. Francis D
    28. Sinangil F
    29. Lee C
    30. Tartaglia J
    31. Robb ML
    32. Haynes BF
    33. Michael NL
    34. Kim JH

    (2012) Magnitude and breadth of the neutralizing antibody response in the RV144 and Vax003 HIV-1 vaccine efficacy trials

    Journal of Infectious Diseases 206:431–441.

    https://doi.org/10.1093/infdis/jis367

    • PubMed
    • Google Scholar
17. 1. Pugach P
    2. Ozorowski G
    3. Cupo A
    4. Ringe R
    5. Yasmeen A
    6. de Val N
    7. Derking R
    8. Kim HJ
    9. Korzun J
    10. Golabek M
    11. de Los Reyes K
    12. Ketas TJ
    13. Julien JP
    14. Burton DR
    15. Wilson IA
    16. Sanders RW
    17. Klasse PJ
    18. Ward AB
    19. Moore JP

    (2015) A native-like SOSIP.664 trimer based on an HIV-1 subtype B env gene

    Journal of Virology 89:3380–3395.

    https://doi.org/10.1128/JVI.03473-14

    • PubMed
    • Google Scholar
18. 1. Rademeyer C
    2. Korber B
    3. Seaman MS
    4. Giorgi EE
    5. Thebus R
    6. Robles A
    7. Sheward DJ
    8. Wagh K
    9. Garrity J
    10. Carey BR
    11. Gao H
    12. Greene KM
    13. Tang H
    14. Bandawe GP
    15. Marais JC
    16. Diphoko TE
    17. Hraber P
    18. Tumba N
    19. Moore PL
    20. Gray GE
    21. Kublin J
    22. McElrath MJ
    23. Vermeulen M
    24. Middelkoop K
    25. Bekker LG
    26. Hoelscher M
    27. Maboko L
    28. Makhema J
    29. Robb ML
    30. Abdool Karim S
    31. Abdool Karim Q
    32. Kim JH
    33. Hahn BH
    34. Gao F
    35. Swanstrom R
    36. Morris L
    37. Montefiori DC
    38. Williamson C

    (2016) Features of recently transmitted HIV-1 clade C viruses that impact antibody recognition: implications for active and passive immunization

    PLOS Pathogens 12:e1005742.

    https://doi.org/10.1371/journal.ppat.1005742

    • PubMed
    • Google Scholar
19. 1. Ringe RP
    2. Ozorowski G
    3. Rantalainen K
    4. Struwe WB
    5. Matthews K
    6. Torres JL
    7. Yasmeen A
    8. Cottrell CA
    9. Ketas TJ
    10. LaBranche CC
    11. Montefiori DC
    12. Cupo A
    13. Crispin M
    14. Wilson IA
    15. Ward AB
    16. Sanders RW
    17. Klasse PJ
    18. Moore JP

    (2017) Reducing V3 antigenicity and immunogenicity on soluble, native-like HIV-1 Env SOSIP trimers

    Journal of Virology 91:e00677-17.

    https://doi.org/10.1128/JVI.00677-17

    • PubMed
    • Google Scholar
20. 1. Sanders RW
    2. Derking R
    3. Cupo A
    4. Julien JP
    5. Yasmeen A
    6. de Val N
    7. Kim HJ
    8. Blattner C
    9. de la Peña AT
    10. Korzun J
    11. Golabek M
    12. de Los Reyes K
    13. Ketas TJ
    14. van Gils MJ
    15. King CR
    16. Wilson IA
    17. Ward AB
    18. Klasse PJ
    19. Moore JP

    (2013) A next-generation cleaved, soluble HIV-1 Env trimer, BG505 SOSIP.664 gp140, expresses multiple epitopes for broadly neutralizing but not non-neutralizing antibodies

    PLoS Pathogens 9:e1003618.

    https://doi.org/10.1371/journal.ppat.1003618

    • PubMed
    • Google Scholar
21. 1. Sarzotti-Kelsoe M
    2. Bailer RT
    3. Turk E
    4. Lin CL
    5. Bilska M
    6. Greene KM
    7. Gao H
    8. Todd CA
    9. Ozaki DA
    10. Seaman MS
    11. Mascola JR
    12. Montefiori DC

    (2014) Optimization and validation of the TZM-bl assay for standardized assessments of neutralizing antibodies against HIV-1

    Journal of Immunological Methods 409:131–146.

    https://doi.org/10.1016/j.jim.2013.11.022

    • PubMed
    • Google Scholar
22. 1. Seaman MS
    2. Janes H
    3. Hawkins N
    4. Grandpre LE
    5. Devoy C
    6. Giri A
    7. Coffey RT
    8. Harris L
    9. Wood B
    10. Daniels MG
    11. Bhattacharya T
    12. Lapedes A
    13. Polonis VR
    14. McCutchan FE
    15. Gilbert PB
    16. Self SG
    17. Korber BT
    18. Montefiori DC
    19. Mascola JR

    (2010) Tiered categorization of a diverse panel of HIV-1 Env pseudoviruses for assessment of neutralizing antibodies

    Journal of Virology 84:1439–1452.

    https://doi.org/10.1128/JVI.02108-09

    • PubMed
    • Google Scholar
23. 1. Tibshirani R

    (1996)

    Regression shrinkage and selection via the lasso

    Journal of the Royal Statistical Society. Series B, Statistical Methodology 1:267–288.

    • Google Scholar
24. 1. Torrents de la Peña A
    2. Julien JP
    3. de Taeye SW
    4. Garces F
    5. Guttman M
    6. Ozorowski G
    7. Pritchard LK
    8. Behrens AJ
    9. Go EP
    10. Burger JA
    11. Schermer EE
    12. Sliepen K
    13. Ketas TJ
    14. Pugach P
    15. Yasmeen A
    16. Cottrell CA
    17. Torres JL
    18. Vavourakis CD
    19. van Gils MJ
    20. LaBranche C
    21. Montefiori DC
    22. Desaire H
    23. Crispin M
    24. Klasse PJ
    25. Lee KK
    26. Moore JP
    27. Ward AB
    28. Wilson IA
    29. Sanders RW

    (2017) Improving the immunogenicity of native-like HIV-1 Envelope trimers by hyperstabilization

    Cell Reports 20:1805–1817.

    https://doi.org/10.1016/j.celrep.2017.07.077

    • PubMed
    • Google Scholar
25. 1. Wu X
    2. Yang ZY
    3. Li Y
    4. Hogerkorp CM
    5. Schief WR
    6. Seaman MS
    7. Zhou T
    8. Schmidt SD
    9. Wu L
    10. Xu L
    11. Longo NS
    12. McKee K
    13. O'Dell S
    14. Louder MK
    15. Wycuff DL
    16. Feng Y
    17. Nason M
    18. Doria-Rose N
    19. Connors M
    20. Kwong PD
    21. Roederer M
    22. Wyatt RT
    23. Nabel GJ
    24. Mascola JR

    (2010) Rational design of envelope identifies broadly neutralizing human monoclonal antibodies to HIV-1

    Science 329:856–861.

    https://doi.org/10.1126/science.1187659

    • PubMed
    • Google Scholar
26. 1. Yu X
    2. Gilbert PB
    3. Hioe CE
    4. Zolla-Pazner S
    5. Self SG

    (2012) Statistical approaches to analyzing HIV-1 neutralizing antibody assay data

    Statistics in Biopharmaceutical Research 4:1–13.

    https://doi.org/10.1080/19466315.2011.633860

    • PubMed
    • Google Scholar

Author details

1. Peter Hraber

   Theoretical Biology and Biophysics, Los Alamos National Laboratory, Los Alamos, United States

   Contribution

   Conceptualization, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing—original draft, Writing—review and editing

   For correspondence

   phraber@lanl.gov

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-2920-4897

2. Bette Korber

   1. Theoretical Biology and Biophysics, Los Alamos National Laboratory, Los Alamos, United States
   2. New Mexico Consortium, Los Alamos, United States

   Contribution

   Conceptualization, Resources, Supervision, Funding acquisition, Validation, Investigation, Methodology, Writing—original draft, Writing—review and editing

   Competing interests

   No competing interests declared

3. Kshitij Wagh

   Theoretical Biology and Biophysics, Los Alamos National Laboratory, Los Alamos, United States

   Contribution

   Conceptualization, Formal analysis, Validation, Investigation, Methodology, Writing—review and editing

   Competing interests

   No competing interests declared

4. David Montefiori

   Department of Surgery, Duke University Medical Center, Durham, United States

   Contribution

   Conceptualization, Resources, Supervision, Funding acquisition, Investigation, Project administration, Writing—review and editing

   Competing interests

   No competing interests declared

5. Mario Roederer

   Vaccine Research Center, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, United States

   Contribution

   Conceptualization, Resources, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing—original draft, Writing—review and editing

   Competing interests

   No competing interests declared

• 1,625

  views

• 179

  downloads

• 15

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.