Improving human papillomavirus (HPV) testing in the cervical cancer elimination era: The 2021 HPV LabNet international proficiency study

Highlights • The 2021 global HPV genotyping proficiency study included 132 laboratories.• An all-time high proportion of datasets were fully proficient: 75%.• High proficiency detected among many laboratories without previous participation.• Laboratories with previous proficiency study participation also improved.


Introduction
In the era of cervical cancer elimination, many laboratories perform Human Papillomavirus (HPV) detection and the number of different assays used has increased exponentially. [1] In 2012 there were already 125 different commercial HPV tests on the global market and in 2020, there were 254 distinct HPV assays and 425 assay variants. [2,3] Accurate and internationally comparable HPV detection is a must for both research and evaluation of HPV vaccination, as well as for screening programs. [4] A recent meta-review of publications validating the different HPV assays found that only 11 HPV DNA assays had published in the peer-reviewed literature with analytical/clinical evaluation. [3] The International HPV Reference Center (IHRC) coordinated the Global WHO HPV Laboratory Network at the outset and has since 2008 coordinated global HPV genotyping proficiency studies to a) assess the proficiency of HPV typing assays when routinely used in laboratories worldwide, b) evaluate the sensitivity and type-specificity of HPV detection of the different HPV assays when routinely used in laboratories worldwide and, c) identify problems with any assays routinely used. [5][6][7][8][9][10] The proficiency panels (PPs) consist of blinded samples with defined amounts of HPV DNA (traceable to International Standards) in a background of human DNA, as well as extraction controls. Laboratories genotype these samples with their standard methods.
In 2019 we found a global decrease in proficiency of HPV genotyping [10] and have therefore increased the ambition level of our international quality assurance program, including issuing of PPs every year. The present report describes the results of the 2021 proficiency study.

2021 proficiency panel (PP)
The 2021 PP included 44 coded samples (40 samples containing one or more purified HPV plasmids diluted in TE buffer (10 mM TRIS-HCl, 0.1 mM EDTA, pH 8.0) with 10 ng/ul of human placental DNA (Sigma-Aldrich no 7011), 1 negative control (TE buffer with 10 ng/µl human placenta DNA) and 3 samples of cell lines used as controls for the DNA extraction step in the testing ( Table 1).
The PP was pre-tested for validation purposes at the Reference Laboratory in Sweden using a modified GP5+/6+ PCR followed by Luminex-based typing and distributed afterwards to laboratories throughout the world. Distribution of panels in November 2021 was performed by Equalis (External quality assessment of laboratory medicine in Sweden, https://www.equalis.se/en/), following the call for participation and requests received. [10] The fee for participation was 1000 Euros for commercial entities and 500 Euros for academic and public health entities. Participants from low and lower-middle income countries could apply for waiving of the fee.
All participants were to determine the genotypes in the panel using their standard assays, using their standard sample input volume. There were no particular demands on the assays used and both commercial assays and in-house assays were allowed. In case a participant used several different HPV detection methods, one PP could be used by each assay and one dataset provided from each method.

Data analysis
Datasets from laboratories submitted until the 28th of February 2022 were accepted and compiled by Equalis and analysed by the IHRC.
Proficiency criteria was established by a WHO consensus meeting 14 years ago, [4] requiring detection of at least 50 IU/5ul of HPV 16 and HPV 18 and 500 GE/5 ul of the other HPV types, in both single and multiple infection. For proficiency, it was also required that no false positive HPV type was detected.
Input with 10 or 15 ul was classified as the same IU/GE content as compared to 5 ul input. The panel also included low concentration samples (5 IU/5 ul for HPV16/18 and 50 GE/5 ul for non-16/18 types) that were intended for educational purposes, training and for providing information on whether the test exceeded the requirements needed for proficient HPV typing. Detection of these low amounts was not required for proficiency.
For assays that did not claim to detect all HPVs included in the PP, evaluation was performed considering only the HPV types that the assay reportedly detected. Consequently, the denominator varies (not all assays targeted all HPV types). For assays reporting results only as aggregated types (e.g. "other"), proficiency was considered as long as the assay detected the HPV types targeted by the method. (e.g.: if an assay only claiming to detect "Other HPV" tested the sample containing HPV 11/33/51/68a and reported that it contained "Other HPV", it was considered proficient).
The results were sent to all participating laboratories that had paid the fee before February 2022.

Results
The 2021 proficiency study was distributed to 144 subscribing laboratories. A total of 132 laboratories returned 211 datasets (every laboratory could submit more than one dataset if several different HPV typing methods were usedone dataset per method). Ninety-six laboratories submitted 1 dataset, 13 laboratories submitted 2 datasets, 4 laboratories submitted 3 datasets, 18 laboratories submitted 4 datasets and 1 laboratory submitted 5 datasets.
There were 126 participating laboratories that provided data on type of laboratory. There were 83 diagnostic and screening laboratories, 26 laboratories performing HPV surveillance, 8 clinical trial laboratories and 9 diagnostic test manufacturers. The annual number of samples analysed for HPV per laboratory varied from 100 to > 100,000 per year, with 15 laboratories performing 10,000 tests or more.
Among datasets that typed for at least one HPV type 158/211 (75%) were 100% proficient for the types claimed to be detected by the test. Of these, 119/158 datasets not only correctly identified the content of all required samples, but also correctly identified the content of the training samples that contained amounts that were lower than required for proficiency (samples that contained only 5IU of HPV16/18 and 50GE of other HPVs). Tests that did not type for all the types in the panel could still be 100% proficient, as the denominator was the number of types claimed to be detected by the test (not the number of types included in the panel).

Proficiency by HPV type
The number of laboratories reporting correct HPV types, with no false positivity, are shown in Table 1. Median percent of 100% proficient datasets for the different coded samples was 94.8% (minimum 75.9%, maximum 99.0%). All datasets showed >90% proficiency for all coded samples containing 50 UI or 500 GE/ 5ul except for HPV 68a, where 75.9% and 78.9% of the datasets had detected HPV 68a at 50 GE/5 ul and 500 GE/5 ul, respectively.

Proficiency by WHO region
Laboratories submitting datasets belonged to 6 WHO regions, with Western Pacific Regional Office (WPRO) showing the highest number of participating laboratories (n = 59), followed by European Regional Office (EURO, n = 49), Pan American Health Organization (PAHO, n = 10), African Regional Office (AFRO, n = 7), Eastern Mediterranean Regional Office (EMRO, n = 5) and South-East Asian Regional Office (SEARO, n = 2).
The proportion of laboratory proficiency including all datasets grouped by WHO region is shown in Fig. 1 and Table 2. WPRO showed the highest number of proficient laboratories (93%) and EURO was the region with the highest number of non-proficient laboratories, with only 49% of laboratories being 100% proficient.

Proficiency by assay used
The different assays used are displayed in Fig. 2 and the number of proficient datasets, false positivity, and HPV targets per assay in Table 3.
A total of 182 datasets were obtained using commercially available tests, with the most commonly assays used being Hybribio 21 array (Hybribio, 39 laboratories), Anyplex II HPV 28 (Seegene, 21 laboratories) and 3 other Hybribio assays (20 laboratories) (  Fig. 1. Proficiency for HPV DNA typing by WHO region. Proficiency criteria were: i) detection at least 50 international units (IU) per 5 ul of HPV 16 and HPV 18, in both single and multiple HPV infections, ii) detection of at least 500 genome equivalents (GE) in 5 ul of the other HPV types (not HPV 16 nor HPV 18) in both single and multiple HPV infections and iii) no false positivity detection. AFRO: African Regional Office, EMRO: Eastern Mediterranean Regional Office, EURO: European Regional Office, PAHO: Pan American Health Organization, SEARO: South-East Asian Regional Office, WPRO: Western Pacific Regional Office. AFRO: African Regional Office, EMRO: Eastern Mediterranean Regional Office, EURO: European Regional Office, PAHO: Pan American Health Organization, SEARO: South-East Asian Regional Office, WPRO: Western Pacific Regional Office.
eight datasets were generated using assays that either did not discriminate specific HPV types or reported results as HPV 16, 18 and "other" HPV types (Hybribio 14 HR, Roche Cobas 4800/6800 test, Abbot Realtime PCR, High risk HPV Screen, Harmonia HPV, and Oncopredict Screen). These datasets were only analyzed for the specific types tested for individually. Twenty-nine datasets were obtained using a variety of in-house assays.
We investigated if there was any specific sample or assay where false positivity was consistently detected. Overall, false positivity appeared to be essentially randomly distributed among samples, indicating that the problem with false positivity was not related to a specific sample nor an assay itself (e.g. cross-reactivity), but rather due to laboratory conditions of use (e.g. cross-contamination).
Comparison of results for laboratories that participated in 2021 as well as in 2008, 2010, 2011, 2013, 2014, 2017 and 2019. In total, 65 laboratories that participated in 2021 had also participated in at least one of the previous proficiency studies. There were 15 laboratories that submitted results in 2021 that had been participating ever since the first PP was issued in 2008, with 10/15 having participated in all eight previous proficiency studies (2008, 2010, 2011, 2013, 2014, 2017, 2019 and 2021). Some laboratories used the same tests during all years, whereas some laboratories had changed at least one of the tests used.
Comparisons of these results were made for each laboratory using the current proficiency criteria for evaluating datasets obtained during all the years. At the outset in 2008 one false positivity had been allowed, but since 2019 no false positivity is required for 100% proficiency. We now retrieved all crude data from previous years and re-calculated proficiency using exactly the same criteria as used since 2019 (no false positivity allowed). The proficiency trend for laboratories participating multiple times is shown in Table 3. The 2021 PP revealed an increase in 100% proficiency (from 25% to 60%) and a decrease in <80% proficiency (from 45.1% to 33.1%) when compared to 2019. Moreover, the highest overall proficiency (75% of datasets) was achieved in 2021 -higher than in any previous proficiency study.

Discussion
We report that the global proficiency in HPV genotyping services had increased in 2021. The increased proficiency is seen both in laboratories that had participated in previous studies as well as in laboratories participating for the first time. This improvement suggests that continuing proficiency testing is helpful to sustain accuracy and to avoid a deterioration in proficiency, as found in 2019. [10] There are three major strengths of our proficiency study. [10] First, the use of the same PP design as in previous PPs enables a directly comparable estimate of the global development of proficiency in HPV genotyping services. Second, the panel contains all 13 HPV types established as oncogenic and the 2 non-oncogenic vaccine-targeted HPV types, reflecting the HPV genotyping requirements that are most important for HPV vaccine research, surveillance, and monitoring, and third, the broad worldwide distribution (132 laboratories worldwide) and variety of different HPV tests used that make generalizability possible.
Weakness of the study is that the PP was composed for evaluating the HPV genotyping services required to support HPV vaccine research and studies monitoring vaccination impact. However, HPV screening may not need to always separate all different HPV genotypes and the sensitivity requirement may also be different. Similar panels with screeningrelevant concentrations of HPV genotypes will be developed to assess and validate the assays (and their use in laboratories) used in cervical screening and triage of high-risk HPV-positive subjects. Another weakness is that we do not have any explanation for the low detectability of HPV68a. Samples were diluted in human DNA as typically found in clinical samples. Further dilution of the samples may have led to a Fig. 2. Type of assay in use for HPV DNA typing by WHO region. AFRO: African Regional Office, EMRO: Eastern Mediterranean Regional Office, EURO: European Regional Office, PAHO: Pan American Health Organization, SEARO: South-East Asian Regional Office, WPRO: Western Pacific Regional Office. Data for AFRO, EMRO and SEARO region are combined.  Operon, HPV screen, SACACE HPV screen, yd-diagnostics, aid-diagnostika, Cephid GeneXpert, CLART 4 Genomica. b Other In-house assays include one laboratory using each of: In-house RFLP, In-house Blot, In-house gel-electroforesis and In-house Mass-array. c All datasets provided using the assay were classified as 100% proficient. d All datasets provided using the assay were classified non-proficient. e Most datasets provided using the assay were classified as non-proficient. f All datasets provided using the assay did not report false positivity. g All datasets provided using the assay reported false positivity. h Most datasets provided using the assay reported false positivity. decrease in proficiency. As these samples were readily detectable by a majority of labs, it is unlikely that the low detectability could be caused by a problem with the samples. The proportion of commercial assays has increased over time, from 57% in 2011, 80% in 2019, and now 86% in 2021 (182/211). Surprisingly, only 13/211 datasets had used any one of the 11 assays that in systematic reviews of the literature had been found to have had published validation data fulfilling all validation criteria (Abbott RealTime High Risk HPV, n = 3; Anyplex II HPV HR Detection, n = 5; BD Onclarity HPV Assay, n = 2; Cobas 4800 HPV Test, n = 3). [11] Both Anyplex II HPV HR Detection and Cobas 4800 showed false positivity in one dataset, and none of the 4 other validated assays were fully proficient in more than 2/3ds of datasets. Only 1/3 HPV assays that have been prequalified by WHO was used by any laboratory (Xpert HPV from Cepheid, used by one laboratory). [12] A common finding, seen also in previous studies, is that the laboratory performing the test has a big impact on the performance of the test itself in particular for certain assays. An example is Anyplex II HPV 28, where 15/21 datasets were fully proficient, but 5/21 datasets were not proficient with 1 to 3 false positive results. An example of assays with robust performance in many laboratories is the assays by Hybribio. These were fully proficient in nearly all datasets (97/98 datasets) from different laboratories, comprising >50% of all fully proficient datasets in the 2021 proficiency study. The widespread use of these assays in laboratories in the WPRO region appears to be a reason why the proficiency was highest in this region. Hybribio was used in 59/132 laboratories, with most of them (44/47) belonging to the WPRO region.
The 2021 Global HPV LabNet HPV DNA proficiency study further enables improving and sustaining sensitivity and specificity of different HPV typing assays. Comparing the results in the 2008, 2010, 2011, 2013, 2014, 2017, 2019 and 2021 global HPV DNA PPs, we can both see that it is possible to achieve a global improvement in proficiency of HPV genotyping services and that regular issuing of PPs is needed for sustaining and improving HPV detection. In the efforts to eliminate cervical cancer, the IHRC will continue to issue PP yearly to promote proficiency in HPV testing. Starting in 2022, we will also be issuing HPV screening panels (with screening-relevant concentrations of the HPV genotypes important for screening) to specifically promote proficiency in HPV screening services.