Improvement in serological diagnosis of pertussis by external quality assessment

Received 30 August 2018; Accepted 18 December 2018; Published 16 April 2019 Author affiliations: Division of Bacteriology, National Institute for Biological Standards and Control (NIBSC), Blanche Lane, South Mimms, Potters Bar, Hertfordshire, EN6 3QG, UK; Immunisation and Countermeasures Division, Public Health England National Infection Service, 61 Colindale Avenue, London NW9 5EQ, UK; University of Turku, Institute of Biomedicine, Department of Microbiology, Virology and Immunology, Kiinamyllynkatu 10, 20520 Turku, Finland; European Centre for Disease Prevention and Control (ECDC), Gustav III:s boulevard 40, 16973 Solna, Sweden; Department of Medical Microbiology, Capital Medical University, No.10 Xi Tou Tiao, You'an Men Wai, Feng Tai District, 100069 Beijing, PR China. *Correspondence: Kevin Markey, kevin. markey@ nibsc. org


InTRoduCTIon
The Gram-negative bacterium Bordetella pertussis is the aetiological agent of whooping cough or pertussis, a contagious respiratory disease [1].Despite the fact that vaccines have existed and been widely used for the prevention of pertussis since the 1940s it remains endemic in all over the world and is one of the most common vaccine-preventable diseases worldwide [2][3][4].In the past decade, significant outbreaks of pertussis have been reported in countries with high vaccine coverage [5].Timely and accurate laboratory diagnosis of pertussis is essential for the treatment, prevention and control of the disease.A broad spectrum of laboratory methods is used for the diagnosis of pertussis.Serological analysis for the presence of anti-pertussis toxin (anti-PT) remains one of the most popular methods for diagnosis of pertussis.Serology involves immunological methods suitable for measuring the levels of B. pertussis-specific antibodies present as response to infection or vaccination [6,7].The most common methods are the ELISA, whereas multiplex immunoassays (MIA) are also used [8][9][10].Diagnostic and research laboratories have tended to either perform in-house or commercial kit ELISAs.A wide number of in-house ELISA protocols and commercially available ELISA kits are available [11].

Background and objectives of ECdC in the pertussis area
The European Centre for Disease Prevention and Control (ECDC) is a European Union (EU) agency with a mandate to operate disease surveillance networks and to identify, assess, and communicate current and emerging threats to human health from communicable diseases [12].
External quality assessment (EQA) is part of quality management systems and evaluates the performance of laboratories by an outside organization, using material that is supplied specially for this purpose.ECDC organizes EQAs for different pathogens in EU/ European Economic Area (EEA) countries.An EQA aims to identify the current performance of the laboratories, identify challenges and areas for improvement in laboratory diagnostic capacities, and to ensure high-quality and comparability of results.
The ECDC has sponsored a number of networks which have, among other activities, arranged EQA schemes for pertussis serology.These networks included EUVAC.NET and currently EUPert-LabNet, which is coordinated by the University of Turku [13,14].

Pertussis serology
The use of methods (e.g.ELISA), which detect antibodies raised against B. pertussis antigens to diagnose infection from human sera, started in the 1980s [7,15].In the 1990s, serology was even more extensively used by research laboratories and vaccine producers for measuring antibody response in clinical trials for acellular pertussis vaccines [16,17].Serological assays routinely measure IgG due to the increased sensitivity and specificity over other classes of immunoglobulins [18].Alongside this, the ability of single-sample serological techniques for diagnosing pertussis became popular [19].In 2000, de Melker et al. [20] demonstrated, that an anti-PT IgG level equal to or above 100 U ml −1 was indicative of current or recent infection.However, there is minimal consensus on the anti-PT IgG titre associated with recent infection.Internationally, values range from 50 to 125 International Units per millilitre (IU ml −1 ), and cut-offs in the region of 70-100 IU ml −1 are used in Europe [8,[21][22][23][24].Initially, purified PT or a mixture of pertussis antigens were used to coat ELISA plates to measure anti-B.pertussis IgG.However, B. pertussis is the only organism to produce PT while other antigens such as FHA and pertactin are cross-reactive with antigens produced by different Bordetella species among others [8].There have been a number of studies to evaluate the performance of ELISA kits and methods [7,15,16,[25][26][27].There have also been attempts to encourage the harmonization of serological assays and reagents for pertussis diagnosis [18] along with guidelines for best practices and standardized approaches [8,28].

Reference sera
The use of sera is vital in epidemiological studies, the serodiagnosis of B. pertussis infection and the evaluation of responses to vaccines.Due to the wide variety of assays used in these studies, a common reference serum is essential for standardization and inter-laboratory comparison.Initially, reference sera prepared by U.S. Centre for Biologics Evaluation and Research (CBER), Food and Drug Administration (FDA) and referred to as US Reference Pertussis Antiserum (Human) Lots 3, 4 and 5 were widely used and played an important role in the standardization of these assays.When stocks of these preparations became diminished two WHO preparations [First International Standard for pertussis antiserum (human)(06/140) and first WHO Reference Reagent for pertussis antiserum (human) (06/142)] were established with IUs based on the FDA preparations [29,30].

METHodS
As part of the EUVAC.NET contract with ECDC, the first collaborative EQA study was organized in July-October 2010 to assess laboratory performance of serological assays for pertussis, to compare in-house references that were being used and to identify any needs for standardization of the serological assays.In this study participants were supplied with two WHO international reference materials, which contained high and low anti-PT IgG antibody concentrations (335 and 106 IU ml-1 respectively) [6].Following this study and as part of the agreement with ECDC, a second EQA collaborative study was held in July-October 2012 [12].Finally, as part of the contract with ECDC, the third EQA collaborative study was organized in February to April 2016 [31].In the latter two studies, the participants received panels of freeze-dried human sera containing anti-PT IgG concentrations that cover the range of titres seen in clinical samples and common cut-off titres.For all three EQAs, the participants were invited to take part as they were the national reference laboratories for their respective EU/EEA Member States.

ECdC/EuVAC.nET collaborative study 2010
Overall, 17 laboratories from 17 different countries took part in the study (Table 1).Of these, nine performed in-house ELISA methods, ten used commercial ELISA kits (one laboratory used two kits) and two of these participants also performed both.
The in-house ELISA methods distinguished the two samples successfully with the correct relative potencies with a good inter-laboratory agreement of geometric coefficients of variation (GCV) of 16%.The potency of the lower concentration sample relative to the high concentration sample was found to be 0.36.This value is close to the relative potency of 0.316, which was observed in the collaborative study performed when these two sera were established as WHO reference materials [29].Purified PT obtained from various sources was used as coating antigen in all the in-house ELISA assays.
The source and nature (native or genetically modified) of PT did not affect the results [6].
Six types of commercial ELISA kits were included in the study (EUROIMMUN, NovaTec, Savyon, Serion, Demeditec and Anilabs).Of the eleven assays performed using kits, only four (36 %) used purified PT as coating antigen.One of these (EUROIMMUN kits) gave results similar to those for the in-house ELISA assays.Serion kits also used only PT.However, the only participant that used this kit got a slightly higher ratio of 0.55.NovaTec kits used plates coated with PT, and FHA and three of the four participants that used them got close to the expected values, but one obtained a ratio of 0.93, which indicates that it was difficult to distinguish the samples.Savyon and Anilabs kits used coatings of whole cell bacteria of B. pertussis and obtained ratios of 0.84 and 0.57, respectively.The former result may indicate that the coating with the whole cell is not suitable.Demeditec kits used PT, FHA and LPS and yielded values of 0.88, again suggesting that anything other than purified PT is unsuitable for coating plates.
Eight out of the nine participants that used in-house ELISA methods included a reference serum.Only one kit (EURO-IMMUN) used a serum sample calibrated against an international standard.Overall, this means that 59% of the total number of assays used in the study could measure anti-PT IgG titres in IU ml −1 .
This study recommended the use of only purified PT as a coating antigen and to calibrate reference sera in international units or against the US reference lot.The study also recognized the urgent need for standardization of commercial ELISA kits along with guidelines on appropriate evaluation tools [6].

ECdC/EuPert-Labnet EQA for Bordetella pertussis serology 2012
A second EQA was organized in 2012 with a panel of five sera samples of different anti-PT IgG concentrations labelled A-E.Concentrations ranged from negative to highly positive with samples also close to the cut-off points used by many laboratories [12].
In total, 21 participants from 20 countries took part in the study.Ten laboratories performed in-house ELISA or MIA with one participant performing two forms of in-house ELISA with different PT coating antigens and conjugates.Twelve laboratories used six commercial ELISA kits (EUROIMMUN, NovaTec, Savyon, Serion, Virotech, and EuroDiagnostics).One of these laboratories performed an in-house ELISA and also used two different commercial ELISA kits.
Unfortunately, when the data were returned to NIBSC for analysis, it was found, that the samples in the panel did not contain the expected values as presented in Table 2.This is most likely due to an error in the preparation of the panel, the reasons for which could not be identified.However, a number of important conclusions could be drawn from the study.There was a good correlation in the results obtained for each sample by participants who used in-house ELISA and MIA.There was an increase in the number of laboratories that used kits with plates coated with PT only (Fig. 1).Once again, all the in-house methods used purified PT as the coating antigen.In total, 13 assay systems used commercial ELISA kits and of these eight (62%) utilized purified PT.The increase is remarkable from the 2010 study where only 36% of the kits used purified PT (Fig. 1).The remaining five kits were coated with PT and FHA.No kits deployed whole cell B. pertussis or a combination of PT, FHA, and lipopolysaccharide (LPS).This increase indicated that the recommendations of previous EQAs were taken on board by end users.
In the 2010 EQA, 59% of participants adopted the use of reference sera.In the 2012 EQA, there was an increase to 18 (75%) out of 24 assay systems utilized reference sera that were calibrated against the WHO or US standards in IU ml -1 or ELISA units per ml.
Regarding the actual results obtained, all laboratories were able to distinguish the sample A (estimated concentration of IgG-PT <2 IU ml −1 ) from the other samples with higher anti-PT IgG concentrations.The majority of laboratories (95%) showed comparable results for each sample and ranked them in the same order based on increasing concentrations of anti-PT IgG.This identical ranking of the samples indicates the satisfactory performance of these laboratories.
Of the six different commercial ELISA kits tested, two of them, Savyon and NovaTec, used mixed PT and FHA.The only laboratory to use the Savyon kit produced results that were considerably higher than what was expected for each reference sera sample in the panel.Of the four laboratories that used NovaTec, the results were non-linear at either the 0.1 and 1.0% level.Another could not differentiate samples C and D and found sample E to be higher than both, which does not correspond to the results obtained by the other participants.A third participant found the same rank order as the other participants, but two out of three replicates produced non-linear results at the 0.1 and 1.0% level.The final laboratory ranked the samples in the order as expected but the differences between samples C, D and E were not as pronounced as the geometric means for the overall study.These unsatisfactory results once again indicated the need to use only purified PT as a coating antigen.
The participants procured purified PT from eight different sources for in-house assays, and there was no particular trend in results observed.Most laboratories used native PT as coating antigen; however, one laboratory used a genetically modified PT that yielded comparable results with other participants.
In general, in-house ELISA methods showed less variability than the commercial kits as evidenced by the GCV % -20-64% for the in-house ELISA compared with 46-77% for commercial ELISA kits.This finding suggested that the standardization of commercial ELISA kits would help improve comparability and harmonization.
The recommendation from the 2012 EQA to laboratories was to use only the purified PT in all immunoassays.The 2012 EQA also recognized the need for regular EQA studies with serum panels of different anti-PT IgG concentrations for validation and standardization of serological assays.Because commercial ELISA kits are increasingly used, guidelines for their use in the serological diagnosis of pertussis should be included in the future activities for standardization in Europe.
The results of this study indicated that increasing numbers of kit manufacturers and diagnostic laboratories adopted the previous EQA recommendations.

ECdC/EuPert-Labnet EQA for Bordetella pertussis serology 2016
More laboratories participated in the 2016 study than in the previous two (n=25) (Table 1) [31].However, one laboratory failed to return any results.It was found that the overall results for each sample in the panel were close to the proposed values (Table 3).It was notable that more participants used commercial ELISA kits in the study compared to previous ones.Again, kits from six different manufacturers were used (EUROIMMUN, NovaTec, Savyon, Serion, Virotech and Novagnost).
There were five different sources of purified PT for in-house methods and once again, did not confound the outcome.There was a difference in the variability in GCV for in-house methods compared to the commercial ELISA kits.The GCV for in-house methods ranged from 11 to 18% whereas the range for commercial ELISA kits was 12-70 %.The GCV of 70% in the kit assays was mainly due to unexpectedly high readings for one sample from two participants.Both of these participants used NovaTec commercial kits, which have plates coated with PT and FHA.The previous EQA also found that the variability in the overall results obtained from participants that used kits was higher than the in-house ELISA methods (46-77%), but such a wide difference was not seen here.The overall findings suggested that the adoption of previous EQA recommendations by manufacturers to better standardize the ELISA kits is leading to improved comparability and harmonization.
The 2010 EQA demonstrated that immunoassays that used purified PT as the coating antigen produced more accurate results and could differentiate the high titre material from the lower.Therefore, it was recommended that all systems should use purified PT [6].All eight laboratories using in-house methods, and 14 of the 16 laboratories that used kits had purified PT as a coating antigen.The overall number of ELISA methods that used purified PT increased to 92% from 79% in 2012 (Fig. 1).In addition, only two kits (both NovaTec) used a mixture of PT and FHA as coating antigens.
Previous EQAs also recommended that all systems should use reference serum calibrated in IU ml −1 and it was found that the number of laboratories that followed this guidance increased with subsequent EQAs.In 2010, only 59% of ELISA assays performed used sera calibrated in IU ml −1 as a reference and this increased to 75% and 92% in 2012 and 2016, respectively.Only the NovaTec kits did not incorporate a reference serum.

dISCuSSIon
Serological methods are becoming increasingly used to diagnose B. pertussis infection that leads to whooping cough.There is a requirement for EQA studies to determine the accuracy of the methods used and to identify areas in which improvements can be achieved to ensure quality and comparability of results between laboratories among all EU/EEA countries.In addition, the increased use of commercial ELISA kits among clinical microbiology laboratories has raised concerns about the varying quality of these kits for measuring anti-PT IgG in human sera [15,25,32].
The ECDC-led EQAs and their recommendations have resulted in substantial improvements in the serological methods used by national reference laboratories in Europe.The improvement in serological assays can be seen by the fact that in 2010, 32% of participants could not differentiate samples of 335 and 106 IU ml −1 anti-PT IgG.Whereas in 2016, the majority of laboratories could differentiate samples more accurately.
More laboratories have adopted the use of reference or control sera calibrated in IU ml −1 , which increased from 59% in 2010 to 92% in 2016.The use of calibrated reference sera is extremely important for the surveillance of pertussis, the evaluation of the effectiveness of vaccines, to facilitate accurate seroepidemiological studies and enable greater comparison of data between countries.
The most recent EQA study in 2016 recommended the use of purified PT (whether in-house methods or kits) for an accurate determination of anti-PT IgG antibodies levels in a serum sample.ECDC also provides training of personnel from the diagnostic laboratories to help improve performance.The recommendation and the training have led to an increase in the number of participating countries/laboratories using ELISA assays with purified PT as a coating antigen -from 65% in 2010 to 92% in 2016.
The improved performance of sero-diagnostic tests will also have an important impact on more accurate surveillance of pertussis and monitoring the effectiveness of vaccination programmes.The manufacturers of commercial ELISA kits may change the reagents they use or produce new ELISA kits according to the guidance and protocols produced by the ECDC/EUPert-LabNet network [28].Monitoring the performance of commercial ELISA kits in EQAs can give an indication of how well they perform relative to other methods and can therefore identify possible areas of improvement.
The international meeting on B. pertussis assay standardization, held in 2007, highlighted the vital requirement of a proficiency panel of human sera for harmonization of methods between laboratories [18].The sera panel that was prepared for the 2016 EQA is now currently available from NIBSC (NIBSC code: 18/146).It can be used for future EQAs on both international and national level or for individual laboratories requiring validation of their existing or new anti-PT IgG ELISA methods.
Our results from three individual EQA studies clearly show the efficacy of EQA studies to improve and harmonize serological pertussis diagnosis and therefore, we suggest that similar studies should be performed to monitor the real performance of individual laboratories.The guidance and EQAs provided by ECDC/EUPert-LabNet also help national reference laboratories to perform similar studies, in their respective countries, among routine microbiology laboratories performing serological diagnosis of pertussis.In addition, performing regular EQA rounds would contribute to maintaining high-quality serodiagnosis of pertussis in EU/ EEA.In addition, it also shows that extending EQA studies to other infectious diseases is recommended.

Fig. 1 .
Fig. 1.Percentage of participant assays that used different combinations of coating antigens in each of the three Bordetella pertussis serology EQAs that were funded by ECDC.

Table 1 .
List of countries that participated in the three Bordetella pertussis serology EQAs that were funded by ECDC

Table 2 .
Proposed human sera reference panel for 2012 EQA and final geometric means obtained from participants that used in-house methods or commercial kits

Table 3 .
Proposed human sera reference panel for 2016 EQA and final geometric means obtained from participants that used in-house methods or commercial kits