Analytical and Clinical Sample Performance Characteristics of the Onclarity Assay for the Detection of Human Papillomavirus

The objective of this study was to determine the result reproducibility and performance of the BD Onclarity human papillomavirus (HPV) assay (Onclarity) on the BD Viper LT platform using both contrived and clinical specimens. Reproducibility was assessed in BD SurePath liquid-based cytology (LBC) medium (SurePath) using contrived panels (HPV genotype 16 [HPV16] positive, HPV18 positive, or HPV45 positive) or clinical specimens (HPV16, -18, -31, -33/58, -45, or -52 positive or HPV negative). In addition, specimens from 3,879 individuals from the Onclarity trial were aliquoted prior to or following cytology processing and tested for HPV.

at greatest risk for high-grade cervical disease or cancer, and nearly 30 countries in Europe utilize HPV testing in some capacity. Most programs currently involve HPV testing as part of cytology triage or cotesting. Countries such as Australia, The Netherlands, Italy, the United Kingdom, and Sweden have transitioned to HPV primary screening with cytology follow-up as necessary (9,10).
The Onclarity assay is performed on samples obtained from liquid-based cytology (LBC) specimens collected using a Cervex-Brush or Cytobrush (or Cytobrush/spatula) device. However, the order of aliquoting for the Onclarity assay can vary based on the respective screening strategy employed by each laboratory. For example, sites utilizing HPV primary screening with cytology triage will perform HPV testing from an initial LBC aliquot (precytology) and use the remaining vial/specimen for cytology. Conversely, sites employing a screening program with primary cytology testing and HPV triage testing will perform cytology testing first, followed by HPV testing from the specimen after cytology processing (postcytology aliquot). Therefore, it is important to establish that the performance of the HPV assay is unaffected by the order in which the aliquot is taken.
Contrived specimens and pooled clinical specimens were utilized to test reproducibility within Onclarity assay runs and between Onclarity assay runs, study sites, operators, reagent lots, and days of operation. In addition, data from pre-and postcytology aliquot specimens were analyzed to determine whether the Onclarity assay performance is impacted by the order in which the sample is aliquoted (i.e., before or after cytology). Finally, from BD SurePath ("SurePath") (Becton, Dickinson and Company, BD Life Sciences-Integrated Diagnostic Solutions, Sparks, MD, USA) vials obtained during the Onclarity trial, Onclarity assay results were compared in specimens obtained using two different collection devices in order to determine whether performance results are affected based on the method of sampling for endocervical specimens.

MATERIALS AND METHODS
Clinical trial population. Women Ն21 years of age (women Ͼ65 years of age were included if they met U.S. Preventive Services Task Force (USPSTF) screening recommendations) were invited to join the Onclarity trial between 2013 and 2015. Initially, 33,858 subjects (across 31 collection sites) were enrolled; the trial population, criteria for inclusion/exclusion, and procedures involving LBC collection, cytology testing, colposcopy/biopsy procedures, and histology examination/diagnosis were described previously (14). By cytology, 30,489 women characterized as negative for intraepithelial lesions or malignancies (NILM) cytology, 1,960 women identified with atypical squamous cells of undetermined significance (ASCUS) cytology, and 1,122 women identified with ϾASCUS cytology (where ASCUS stands for atypical squamous cells of undetermined significance) were included in the baseline data from the Onclarity trial. The study was approved by institutional review boards at each study site, and written informed consent was obtained prior to any trial-related procedures; this study was conducted according to the principles set forth by the Declaration of Helsinki and good clinical practice, and this report was prepared according to STARD (Standards for Reporting of Diagnostic Accuracy) guidelines for reporting diagnostic accuracy.
Preparation for clinical reproducibility, pre-and postcytology aliquots, and collection device experiments. For reproducibility testing, contrived panel members were prepared using SiHa, HeLa, and MS751 transformed cell lines that express HPV16, -18, and -45, respectively. Aliquots from each cell panel preparation were added to an HPV-negative SurePath clinical matrix to yield high-negative specimens (C 5 [specimens called positive approximately 5% and negative 95% of the time]), low-positive specimens (C 95 [specimens called positive approximately 95% and negative 5% of the time]), and moderate-positive specimens (3ϫ C 95 [specimens approximately three times above the C 95 level and expected to be positive 100% of the time]). These determinations were made based on the assay cycle threshold (C T ) values relative to the clinical cutoff point (C 95 ) associated with the assay. Pooled clinical specimens positive for HPV16, -18, -45, -31, -33/58, or -52 were diluted (with the HPV-negative clinical specimen matrix) to a detection level close to the C 95 (the clinical cutoff). Negative panel members were created by pooling high-risk-HPV-negative clinical specimens. All panel members were stored at Ϫ20°C prior to Onclarity assay testing. Standard deviations and coefficients of variability for PCR mean cycle times within a run, between runs, between operators, between sites, between reagent lots, and between days (see Fig. S1 in the supplemental material) were all factors used as outcome measures of reproducibility.
During the Onclarity trial, endocervical specimens were collected using a Rovers Cervex-Brush (Rovers Medical Devices, The Netherlands) or a Cytobrush Plus GT and a Pap Perfect plastic spatula ("Cytobrush/spatula") (Cooper Surgical, Inc., Trumbull, CT, USA) and stored/transported in SurePath LBC specimen vials. Clinical specimens were processed (as described below) and utilized for HPV testing via the Onclarity assay on the BD Viper LT system ("Viper LT") (Becton, Dickinson and Company, BD Life Sciences-Integrated Diagnostic Solutions, Sparks, MD, USA).
For precytology and postcytology specimens, central laboratory personnel vortexed the SurePath LBC specimen and manually aliquoted 0.5 ml of the specimen into an HPV LBC diluent tube (precytology aliquot). Aliquoting from SurePath LBC specimen vials to diluent tubes was performed in the same order as the specimen vials were received. Following the removal of the 0.5-ml precytology aliquot, 8.0 ml of the specimen was removed from the SurePath LBC vial, and a cytology slide was processed (using the BD PrepMate/PrepStain system; Becton, Dickinson and Company, BD Life Sciences-Integrated Diagnostic Solutions, Sparks, MD, USA) according to the manufacturer's instructions. A final 0.5-ml aliquot from residual fluid in the SurePath LBC vial was manually transferred into a second HPV LBC diluent tube (postcytology aliquot). Thus, pre-and postcytology aliquot diluent tubes were obtained from the same SurePath LBC specimen vials; both diluent tubes were sent to one of four laboratories that ran Viper LT testing (Fig. S2). There was a minimal delay for the postcytology aliquot specimens while cytology slides were prepared; this was within the validated room-temperature storage time. Overall, 3,879 SurePath vials were utilized in the study to provide pre-and postcytology aliquot pairs for HPV testing and analysis.
Sample processing for HPV testing. The details for HPV testing with the Onclarity assay on the Viper LT system using LBC specimens were described previously (16,17). Briefly, Onclarity uses three processing steps: (i) an aliquoted, collected specimen matrix in SurePath medium is vortexed and prewarmed; (ii) the nucleic acids are extracted using BD Fox extraction (Becton, Dickinson and Company, BD Life Sciences-Integrated Diagnostic Solutions, Sparks, MD, USA) that involves automated matrix homogenization, cell lysis, binding, and elution of DNA; and (iii) real-time PCR amplification of both HPV E6/E7 and human ␤-globin (HBB) target DNA sequences is performed on the Viper LT system. TaqMan DNA probes (Thermo Fisher, Pittsburgh, PA, USA) include a fluorescent dye at the 5= end and a quenching molecule at the 3= end of the oligonucleotide. Three individual PCR tubes (G1, G2, and G3) collectively detect 14 high-risk HPV genotypes (6 individual genotypes, 16,18,31,45,51, and 52, and three groups containing 8 genotypes, 33/58, 35/39/68, and 56/59/66). The human beta globin gene served as the internal control for each PCR across all three PCR tubes.
Data collection and analysis. For reproducibility testing, three test sites analyzed panels, testing one panel in duplicate (once per operator) daily, for 9 days. Three different reagent lots were utilized: one lot per 3 days of testing. Panel members were randomized, and technical staff were blind to genotypes in each panel member. A total of 162 results (54 per testing site) were expected for each panel member. Percent agreement (with the accompanying lower and upper 95% confidence intervals) analyses were performed for high-negative, low-positive, and moderate-positive contrived specimens. The acceptance criterion for HPV assay performance during testing of panel members was predetermined: for lowpositive specimens, it was 94%, and for moderate-positive specimens, it was 98% (Table S1). For clinical specimen analysis, specific mean C T scores (between 34.2 and 38.3 for HPV16 and between 29.6 and 34.2 for the other 13 genotypes) were required to ensure that genotypes were being detected in proximity to the clinical cutoff. The limit of detection around the clinical cutoffs for HPV16 (C T value of 38.3) is around 1,500 viral genome copies/ml of undiluted SurePath medium; for the other 13 genotypes (C T value of 34.2), it ranges from 3,000 to 10,000 viral genome copies/ml. Additional information regarding this issue is available in the product's information-for-use document (18).
For pre-versus postcytology aliquot comparison, positive, negative, and overall agreements were determined using the precytology aliquot result to define positive and negative. Mean (with lower and upper 95% confidence intervals) pre-and postcytology aliquot C T scores, including mean differences between the two, were calculated, and statistical comparison was performed using a paired t test. Linear regression was performed for high-risk HPV genotype detection between pre-and postcytology aliquot specimens.
Data for comparison of collection devices were generated at four testing sites in the United States from a precytology aliquot. HBB C T scores, HPV C T scores, and high-risk HPV positivity rates were analyzed in three intended-use populations (ASCUS, Ն21 years of age; NILM, Ն30 years of age; and primary screening, Ն25 years of age) and different age groups (21 to 24, 25 to 29, 30 to 39, 40 to 49, and Ն50 years of age). The mean HBB C T score was calculated by averaging each specimen's three internal control C T score results. The HPV C T score was calculated by selecting the strongest C T score from nine channels, excluding subjects without an HPV C T score result. The HBB and HPV C T scores were compared using a two-sample t test. The P value that was determined using the Satterthwaite approximation for degrees of freedom was reported. To test whether HPV positivity rates were different between the two collection devices, Fisher's two-sided exact test was performed.

RESULTS
Onclarity assay reproducibility. For reproducibility testing, contrived specimens were created using cells expressing HPV16 (SiHa), HPV18 (HeLa), and HPV45 (MS751) to spike an HPV-negative clinical specimen matrix at prespecified low-and moderatepositive concentrations. As shown in Fig. 1 (see also Table S2 in the supplemental material), the Onclarity assay reported results for HPV16, -18, and -45 that were all above 95% agreement within the low-positive panels and near 100% for the moderatepositive panels (both compared to the expected results). For the pooled HPV highnegative clinical panels, 91.6% of the samples were negative for HPV16, whereas 100% of the HPV18 and HPV45 samples returned a correct result of negative. For pooled clinical specimens positive for HPV16, -18, -45, -31, -33/58, or -52, the reproducibility for the mean C T score met the acceptance criteria; the overall standard deviations and percent coefficients of variation ranged from 0.87 to 1.86 and 2.9% to 5.6%, respectively, with the greatest variation being observed within replicates on the same instrument run (Table 1). HPV-negative samples (HPV-negative clinical matrix or HPVnegative cell line suspended in SurePath LBC medium) were all reported as negative (100% had C T values above 38.3 on the HPV16 channel and 34.2 for channels relative to the other eight HPV results) by Onclarity (Table 1). , and -45 were tested with the Onclarity assay. The contrived specimens were prepared at concentrations categorized as "low positive," "moderate positive," and "high negative"; all three are characterized relative to the clinical cutoff. Results from the Onclarity assay for each of the three contrived sample groups were compared to the expected results. An HPV-negative group was included with the high-negative contrived sample run.  Fig. 2, with those from the precytology aliquot on the x axis and those from the postcytology aliquot on the y axis. Although there was a slight difference in the distribution of C T scores between pre-and postcytology aliquot groups, the results corresponding to precytology aliquots and postcytology aliquots were linear and represented a one-to-one correlation. Table 2 shows four categories (ASCUS, ϾASCUS, NILM, and any cytology), which correspond to cytology/HPV triage (ASCUS) for women Ն21 years of age, cotesting (NILM) for women Ն30 years of age, and the primary screening population (any cytology) for women Ն25 years of age. Positive and negative percent agreements were high for all cytology categories, and the overall percent agreement between the preand postcytology aliquot specimens was Ͼ98% for all cytology categories (   concordance rates for positive results from the ASCUS, ϾASCUS, NILM, and any cytology (Ն25 years of age) groups, respectively. In addition, concordance rates for the postcytology aliquot specimens compared to the precytology aliquot specimens were 100% (129/129), 90.5% (19/21), 98.9% (2,431/2,457), and 98.9% (3,052/3,087) for negative results from the ASCUS, ϾASCUS, NILM, and any cytology (Ն25 years of age) groups, respectively. The majority of the discordant results were from women with NILM cytology, and the discordant results also split across the two sample types (precytology aliquot positive/postcytology aliquot negative and precytology aliquot negative/postcytology aliquot positive) (Table S3). Approximately 85% of the discordant results were close to the clinical cutoff of the assay (data not shown).
Mean C T scores were determined for Onclarity results from specimens that were positive for any high-risk HPV genotype (n ϭ 773) or the individual genotype HPV16 (n ϭ 77), HPV18 (n ϭ 34), or HPV45 (n ϭ 42) ( Table 3). Pre-and postcytology aliquot mean C T scores were close across all four test groups, with the mean difference (postcytology aliquot Ϫ precytology aliquot) being no greater than 0.31 cycle from zero. Statistical analyses revealed no significant difference between the mean C T scores from pre-and postcytology aliquot specimens for any of the genotype categories.
Onclarity results based on specimen collection device. Onclarity performances were compared following collection with either the Cervex-Brush or the Cytobrush (or Cytobrush/spatula) in three screening populations: ASCUS, Ն21 years of age (n ϭ 989 for Cervex-Brush and n ϭ 964 for Cytobrush); NILM, Ն30 years of age (n ϭ 11,145 for Cervex-Brush and n ϭ 11,139 for Cytobrush); and primary screening, Ն25 years of age (n ϭ 14,858 for Cervex-Brush and n ϭ 14,654 for Cytobrush). To compare Onclarity performances for each collection device, the C T values were averaged for all samples with a signal (C T Ͻ 40) on the Viper LT system (n ϭ 427 and n ϭ 424 for Cervex-Brush and Cytobrush, respectively, in the ASCUS population; n ϭ 1,427 and n ϭ 1,390 for Cervex-Brush and Cytobrush, respectively, in the NILM population; and n ϭ 2,637 and n ϭ 2,586 for Cervex-Brush and Cytobrush, respectively, in the primary screening population) (Table S4). No significant difference was observed between the Cervex-Brush and the Cytobrush (or Cytobrush/spatula) across the three screening populations or by age group (Fig. 3a and Table S4). In addition, no significant difference was observed between the Cervex-Brush and Cytobrush (or Cytobrush/spatula) C T scores related to the detection of the internal control (HBB gene) (Table S4 and Fig. S3). The HPV positivity rates (for those samples with signals of Յ38.3 for the HPV16 channel and Յ34.2 for the other eight HPV channels) for specimens collected with the Cervex-Brush and Cytobrush (or Cytobrush/spatula) devices were not significantly different across all three screening populations. In addition, positivity rates with both collection devices tended to decrease with increasing age in the primary screening population (Fig. 3b and Table 4). Positivity rates were not significantly different between devices when results were stratified by age (21 to 24, 25 to 29, 30 to 39, 40 to 49, and Ն50 years of age) (Table 4).

DISCUSSION
The results presented here demonstrate the high reproducibility of Onclarity (within run, between runs, between operators, between sites, between reagent lots, and between days). Onclarity met reproducibility criteria for contrived specimens containing individual genotypes 16, 18, and 45 and for pooled clinical specimens positive for either HPV16, -18, -45, -31, -33/58, or -52. The overall agreement of the results from the Onclarity assay using precytology aliquot and postcytology aliquot samples for each of the subject populations was above 98%, with the lower bound of the 95% confidence interval being Ն93%. Finally, HPV positivity rates and HPV mean C T scores, both overall and when stratified by age groups, were not statistically different for the two collection devices (Cervex-Brush or Cytobrush [or Cytobrush/spatula]) investigated here (18). Results from the contrived HPV16 specimens showed a slightly lower percent agreement than the expected result for HPV16 high-negative specimens. As shown in Table S2 in the supplemental material, the majority of the discordance involving the high-negative HPV16 contrived specimens was based on a difference in one location (site 3) and one lot (lot 2). It is not clear that these two instances represent a true depiction of Onclarity assay performance for differentiating HPV16-negative specimens from HPV16-positive specimens around the cutoff. The clinical cutoff C T value for HPV16 (38.3) is approximately 4 cycles higher than that for the other eight Onclarity results (34.2), which may explain the low reproducibility of results for the HPV16 high-negative specimens relative to the other genotype results. However, the Onclarity assay has been clinically validated for HPV16 detection, and previous results for HPV16 from screening populations have demonstrated good specificity and positive predictive values for the detection of the individual HPV16 genotype (5,11,15,(19)(20)(21)(22)(23)(24). In addition, the highly reproducible results observed here for the Onclarity assay regarding contrived and pooled clinical specimens are consistent with previous work. Ejegod and colleagues demonstrated high reproducibility with good intralaboratory agreement (98.6%) and kappa value (0.967) and good interlaboratory agreement (98.4%) and kappa value (0.962) for Onclarity assay-positive/negative results from specimens collected in PreservCyt LBC medium in a subset of an English screening population (12). Similarly, Ejegod and colleagues observed good intra-and interlaboratory reproducibility with the Onclarity assay from specimens collected in SurePath LBC medium from a Danish population (13). In this study, the greatest variation for pooled clinical specimen results was observed within Onclarity assay runs. This was not unexpected as LBC specimens are inherently nonhomogeneous. They are composed of sheaths of sloughed-off, exfoliated cells that are stored in a fixative, which can lead to clumping. For viral signals, this is further exacerbated by the focal nature of HPV infections, often representing just a small fraction of the total cell population in a specimen. All other factors, including between runs, between operators, between sites, between reagent lots, and between days, showed relatively low variation in results compared to the within-run results.
In addition to the Ͼ95% agreement with the expected results for HPV16-, HPV18-, and HPV45-positive contrived specimens, the mean C T scores of individual results for HPV16, -18, -45, -31, -33/58, and -52 from HPV-positive pooled clinical specimens also demonstrate the reproducibility of the Onclarity assay. In addition, 100% of HPVnegative specimens (clinical matrix only) were associated with C T values above the cutoff for a positive result (38.3 on the HPV16 channel and 34.2 for channels relative to the other eight HPV channels). This reproducibility is important as countries in North America, Europe, Australia, and Asia continue to consider extended and full genotyping as a triage approach to improve risk detection for high-grade cervical disease during HPV primary screening. Publications from both the Onclarity clinical trial and Kaiser Permanente Northern California have previously demonstrated the potential benefit of extended genotyping to identify either those with NILM cytology as being at highenough risk for a referral for colposcopy (e.g., those with NILM cytology and positive for HPV16 or -31) or those with ASCUS/low-grade squamous intraepithelial lesion (LSIL) as being of low-enough risk to return for follow-up as opposed to a referral for immediate colposcopy (e.g., those with ASCUS/LSIL cytology and positive for HPV56) (21,22,32). A recent systematic review outlines further evidence for extended/full genotyping as an effective means for triage in both U.S.-based populations and populations outside the United States (37).
As many HPV assays are nucleic acid amplification based, precytology aliquot specimens are typically preferred for HPV testing prior to LBC processing. However, current cervical cancer screening recommendations, both inside and outside the United States, vary based on the age of the screening population (among other factors, including prior screening/treatment status). Approaches to cervical cancer screening also vary from country to country. In Europe, for example, approximately 55% of countries utilize cytology as the primary screening modality, with 45% of countries utilizing some combination of cytology and HPV testing (9). Depending on the country or region, specimens for HPV testing could be aliquoted either before or after the specimen is processed for cytology. Therefore, it is important to understand how precytology aliquot and postcytology aliquot specimens may or may not vary for HPV assay performance. The overall agreement of the results from the Onclarity assay using precytology aliquot and postcytology aliquot specimens, representing three cervical cancer screening populations, was high. However, the discordant results that were observed are not unexpected, especially in samples close to the cutoff of the Onclarity assay. As discussed above, LBC specimens are inherently nonhomogeneous, which warrants confirmation of pre-and postcytology analyses to confirm within-specimen consistency. Although specimens are not routinely tested twice, laboratories may test specimens either before or after cytology, depending on their preferred workflow and standard-of-care screening paradigm (e.g., cotesting versus HPV primary screening). In addition, a lower agreement was observed for the pre-and postcytology results in the NILM cytology group. NILM cytology positive for HPV a priori represents early or receding infections; thus, enrichment is likely occurring in this cytology group for HPV-positive results that are close to the clinical cutoff of the assay. Qualitative HPV assays show more variability at low infection levels.
Here, the collection device had no overall impact on the HPV result. In addition, there was no observed effect of the collection device type across age groups, and therefore, either collection device should be effective for different screening popula-tions (ASCUS triage, Ն21 years; cotesting, Ն30 years; and Ն25 years). The squamocolumnar junction (SCJ) is an anatomical area in which cellular transformation occurs at a high rate and is a common region in which abnormal cells develop. With age, the cervical transformation zone (including its distal edge, the SCJ) recedes into the cervical canal (27), which renders LBC collection from the SCJ challenging. Here, the choice of collection device did not impact the ability to detect HPV for the Ն40and Ն50-year age groups.
Limitations. Clinical specimens used in this study were obtained from the Onclarity trial, a large cervical cancer screening trial conducted in the United States, which has been described previously. Therefore, some aspects of bias or imprecision associated with the experimental design or procedures related to the Onclarity trial may apply to these analyses. These would include some types of partial verification bias when stratifying results by age or cytology result. This was addressed for results from the Onclarity trial previously through a statistical methodology to adjust for verification bias, which was not conducted during stratification by cytology results for pre-and postcytology and collection device analyses. In addition, classification bias due to a lack of a true reference for pre-and postcytology and collection device analyses on HPV detection may have led to some inaccuracies in our results here. Some form of analytic bias could have occurred here, especially between study sites, which has not been explained and may have impacted our results (for example, results for the highnegative HPV16 specimens). Finally, as discussed above, regarding the HPV16 highnegative results, bias could have affected the accuracy of our results for HPV16 compared to the other eight HPV results as the difference in the HPV signal-negative (38.3 Ͻ C T Ͻ 40) and signal-positive (C T Յ 38.3) C T values is smaller (the cutoff is closer than the limit of detection) than that for the other eight HPV results: signal negative, 34.2 Ͻ C T Ͻ 40; signal positive, C T Յ 34.2. Finally, histological outcomes were not used here to determine whether Onclarity assay results involving clinical specimens corresponded to performance compared to histological outcomes. However, the objective of this study was to determine the analytical performance of the Onclarity assay using clinical and contrived specimens, irrespective of the ability of the Onclarity assay to detect cancer or precancer. The clinical performance of the Onclarity assay, compared to histology as a reference, has been described extensively elsewhere (11,15,20,23).
Conclusion. Overall, the results here characterize the impact of preanalytical activities on Onclarity assay reproducibility and provide evidence for the potential flexibility of the Onclarity assay within different workflows during cervical cancer screening. This includes sample collection devices, aliquoting order, and other laboratory workflow practices. Regardless of each of these factors, the results obtained with the Onclarity assay on the Viper-LT system were robust and reproducible.

SUPPLEMENTAL MATERIAL
Supplemental material is available online only. SUPPLEMENTAL FILE 1, PDF file, 0.4 MB.