Reduced Relative Sensitivity of the Elecsys SARS-CoV-2 Antigen Assay in Saliva Compared to Nasopharyngeal Swabs

Early identification and isolation of SARS-CoV-2-infected individuals is central to contain the COVID-19 pandemic. Nasopharyngeal swabs (NPS) serve as a specimen for detection by RT-PCR and rapid antigen screening tests. Saliva has been confirmed as a reliable alternative specimen for RT-PCR and has been shown to be valuable for diagnosing children and in repetitive mass testing due to its non-invasive collection. Combining the advantages of saliva with those of antigen tests would be highly attractive to further increase test capacities. Here, we evaluated the performance of the Elecsys SARS-CoV-2 Antigen assay (Roche) in RT-PCR-positive paired NPS and saliva samples (N = 87) and unpaired NPS (N = 100) with confirmed SARS-CoV-2 infection (Roche cobas SARS-CoV-2 IVD test). We observed a high positive percent agreement (PPA) of the antigen assay with RT-PCR in NPS, reaching 87.2% across the entire cohort, whereas the overall PPA for saliva was insufficient (40.2%). At Ct values ≤ 28, PPA were 100% and 91.2% for NPS and saliva, respectively. At lower viral loads, the sensitivity loss of the antigen assay in saliva was striking. At Ct values ≤ 35, the PPA for NPS remained satisfactory (91.5%), whereas the PPA for saliva dropped to 46.6%. In conclusion, saliva cannot be recommended as a reliable alternative to NPS for testing with the Elecsys Anti-SARS-CoV-2 Antigen assay. As saliva is successfully used broadly in combination with RT-PCR testing, it is critical to create awareness that suitability for RT-PCR cannot be translated to implementation in antigen assays without thorough evaluation of each individual test system.


Introduction
Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) infection is routinely diagnosed by reverse-transcription polymerase chain reaction (RT-PCR) in nasopharyngeal swabs (NPS). In early infection, NPS are considered the optimal material for detection [1]. However, NPS collection requires trained personnel, exposes the personnel

Study Population
Probed specimen were leftover from routine diagnostics and a previous study [21] approved with an ethics waiver by the Zurich Cantonal Ethics Commission (Req-2020-00398). No additional personal data beyond the usual information on symptoms and duration required by the Swiss Federal Office of Public Health for all SARS-CoV-2 tests in Switzerland were collected.
For nasopharyngeal sampling, conventional swab and virus transport medium (VTM)/universal transport medium (UTM) were used. Transport media included in-house VTM (HEPES, DMEM, FCS, antibiotics, antimycotics) and Virus Preservative Medium (Improviral) for paired PCR-positive NPS; Liquid amies preservation medium (Copan) for unpaired PCR-positive NPS; in-house VTM for paired PCR-negative NPS. Saliva collection was conducted as previously described [21]. Briefly, individuals were asked to clear the throat thoroughly and collect saliva up to three times into an empty saliva collection tube to collect 0.5-1 mL (approx. a teaspoon full). Immediately after saliva collection, VTM was added to the crude saliva and the content mixed through gentle tilting. Saliva was collected directly after NPS for paired analysis and both specimens were immediately sent for SARS-CoV-2 RT-PCR testing. Residual material was stored at −80 • C until use in the present study.

Specimen
RT-PCR tested (Roche cobas SARS-CoV-2 IVD test), cryo-preserved (−80 • C) NPS and saliva specimens were included. Paired SARS-CoV-2 NPS and saliva included positive (N = 87) and negative tested samples (N = 220) collected in the framework of a prior study that conducted a prospective head-to-head comparison of the specimen by RT-PCR [21]. In addition, SARS-CoV-2 positive NPS samples from routine diagnostic testing were included (N = 100).

Quantitative SARS-CoV-2 PCR
NPS and saliva were processed identically using the procedures established for NPS in the diagnostics laboratory of the Institute of Medical Virology, University of Zurich. A total of 500 µL of NPS or saliva in VTM were diluted in 500 µL of Nuclisens easyMAG Lysis Buffer (bioMérieux, Marcy-l'Étoile, France), centrifuged (2000 rpm, 5 min), and analyzed with the cobas SARS-CoV-2 IVD test (Roche, Basel, Switzerland) on a Cobas 6800. RT-PCR testing for paired NPS and saliva was done in parallel on the same day as part of a previously published study [21]. SARS-CoV-2 detection was further quantified using SARS-CoV-2 Frankfurt 1 RNA as a calibrator (European Virus Archive, 004N-02005) allowing to report both Ct and genome equivalents. E-gene Ct values were used for comparison. In one NPS sample, the E-gene Ct value was considerably higher than the ORF1-gene Ct value (37.31 vs. 26.8); in this case, the ORF1-gene Ct value was used for the analysis.

SARS-CoV-2 Antigen Assay
Samples were thawed at room temperature and analyzed with the Elecsys SARS-CoV-2 Antigen assay (Roche, Basel, Switzerland) on a cobas e 411 analyzer (Roche, Basel, Switzerland) according to the manufacturer's instructions. Briefly, Elecsys SARS-CoV-2 Antigen is an electrochemiluminescence immunoassay, which detects the nucleocapsid (N) antigen of SARS-CoV-2. The assay uses an antibody sandwich principle. Results are recorded as the Cutoff Index (COI).
Saliva samples were treated identically to NPS. Single measurements were conducted.

Data Analysis
Positive Percent Agreement (PPA), Negative Percent Agreement (NPA), and Overall Percent Agreement (OPA) between the Elecsys Anti-SARS-CoV-2 Antigen assay and the cobas SARS-CoV-2 IVD test were calculated. PPA was determined and stratified by Ct values (PPAoverall and PPA < Ct). PPA, NPA, and OPA were calculated using R (version 4.0.2); 95% confidence intervals (CI, two-sided) were calculated with the epiR package (version 1.0.15). Analysis of correlation (Spearman correlation) between the E-gene Ct value and antigen assay Cutoff Index (COI) and linear regression analysis were performed with GraphPad Prism (version 8.4.3). p values < 0.05 were considered statistically significant. All raw data are available in Supplemental Table S1.
To assess the specificity of the Roche Elecsys Anti-SARS-CoV-2 Antigen assay, we next determined the Negative Percent Agreement (NPA) and Overall Percent Agreement (OPA) of the Roche Elecsys Anti-SARS-CoV-2 Antigen assay in reference to the cobas SARS-CoV-2 IVD test. NPA was 100% and 99.5% for NPS and saliva, respectively ( Table 3). The lower NPA in saliva was due to a single saliva sample that yielded a positive result (COI = 1.41). A repeat analysis of this sample retrieved a negative result (COI = 0.612). As all other samples were measured only once, the first positive result was considered for analysis. In sum, the Elecsys Anti-SARS-CoV-2 Antigen assay showed a high specificity both in NPS and saliva. The Overall Percent Agreement (OPA) was 94.1% and 82.7% for NPS and saliva, respectively (Table 3).

Low Qualitative Agreement of Antigen Measurements between Paired NPS and Saliva
To assess the capacity of the antigen assay to detect SARS-CoV-2 more directly, we next determined the agreement between the antigen assay results obtained for NPS and saliva in the paired sample cohort. This analysis yielded very low percentages for PPA (41.3%), NPA (66.7%), and OPA (44.8%), underlining the limited performance of saliva testing with the antigen assay (Table 4).

Lower Quantitative Agreement of the Elecsys Anti-SARS-CoV-2 Antigen Assay with RT-PCR Diagnosis in Saliva Than in NPS
We next evaluated the quantitative agreement of the antigen assay and RT-PCR for both NPS and saliva specimens. Antigen measurements (COI values) and RT-PCR (E-gene Ct values) correlated generally well in both specimen types ( Figure 1A); however, saliva yielded a lower correlation coefficient compared to NPS ( Figure 1B). a matrix effect of saliva that affects detection in the antigen assay at equal sensitivity as in NPS. We previously reported that Ct values were on average 4.87 higher in saliva than in the corresponding NPS, which corresponds to, on average, 30-fold lower viral loads in saliva [21]. This was also evident in the paired NPS-saliva cohort analyzed in the present study, where we observed, on average, 4.6 higher Ct values in saliva (data not shown).

Low Capacity of Saliva Antigen Testing in Clinical Diagnosis
The current gold standard for SARS-CoV-2 diagnosis remains RT-PCR detection in NPS, and the sensitivity of other test formats, including antigen tests, are evaluated in reference to it. For SARS-CoV-2 antigen detecting rapid diagnostic tests (Ag-RDTs) of respiratory specimens, the WHO defines as the minimum performance requirements a sensitivity of ≥80% and a specificity of ≥97% compared to a nucleic acid amplification test (NAAT) reference assay (https://www.who.int/publications/i/item/antigen-detection-inthe-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays, accessed on 12 May 2021). Considering that saliva contains less virus than NPS, the performance of saliva antigen testing needs to be put into relation to corresponding levels of the virus in NPS to yield relevant information.
We therefore assessed the diagnostic agreement between NPS RT-PCR as a reference and the paired saliva antigen assay results. The analysis highlighted the markedly reduced clinical performance of saliva antigen testing. PPA overall was insufficient (40.2%) ( Table 5). Even for samples with Ct values (of the corresponding NPS) ≤ 26, PPA Linear regression of only positive antigen tests (COI values ≥ 1) yielded different Egene Ct cutoffs in NPS vs. saliva ( Figure 1C). This shift of 1.45 Ct could potentially indicate a matrix effect of saliva that affects detection in the antigen assay at equal sensitivity as in NPS.
We previously reported that Ct values were on average 4.87 higher in saliva than in the corresponding NPS, which corresponds to, on average, 30-fold lower viral loads in saliva [21]. This was also evident in the paired NPS-saliva cohort analyzed in the present study, where we observed, on average, 4.6 higher Ct values in saliva (data not shown).

Low Capacity of Saliva Antigen Testing in Clinical Diagnosis
The current gold standard for SARS-CoV-2 diagnosis remains RT-PCR detection in NPS, and the sensitivity of other test formats, including antigen tests, are evaluated in reference to it. For SARS-CoV-2 antigen detecting rapid diagnostic tests (Ag-RDTs) of respiratory specimens, the WHO defines as the minimum performance requirements a sensitivity of ≥80% and a specificity of ≥97% compared to a nucleic acid amplification test (NAAT) reference assay (https://www.who.int/publications/i/item/antigen-detectionin-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays, accessed on 12 May 2021). Considering that saliva contains less virus than NPS, the performance of saliva antigen testing needs to be put into relation to corresponding levels of the virus in NPS to yield relevant information.
We therefore assessed the diagnostic agreement between NPS RT-PCR as a reference and the paired saliva antigen assay results. The analysis highlighted the markedly reduced clinical performance of saliva antigen testing. PPA overall was insufficient (40.2%) ( Table 5). Even for samples with Ct values (of the corresponding NPS) ≤ 26, PPA was only 47.4%. In line with these data, analysis of correlation between COI values of saliva and Ct values of the corresponding NPS yielded a very low correlation coefficient ( Figure 2). Thus, saliva testing with the Elecsys Anti-SARS-CoV-2 Antigen assay has overall a low capacity in clinical diagnosis.    The study was designed to assess the analytical performance of saliva antigen testing. While an assessment of the performance of the antigen test in saliva at different disease stages would require a dedicated study protocol and enrollment, we had basic data on Microorganisms 2021, 9, 1700 8 of 13 symptom status available for our cohort ( Table 1) that allowed a first analysis (Figure 3). Comparing diagnostic performance of RT-PCR and the antigen test in paired NPS and saliva samples that tested positive in symptomatics (N = 77) and asymptomatics (N = 6), we observed the expected lower sensitivity of the antigen test in samples with lower viral load (higher Ct value) irrespective of disease pattern and duration (Figure 3). axis) was analyzed for saliva samples (N = 87, p = 0.0002).
The study was designed to assess the analytical performance of saliva antigen testing. While an assessment of the performance of the antigen test in saliva at different disease stages would require a dedicated study protocol and enrollment, we had basic data on symptom status available for our cohort ( Table 1) that allowed a first analysis (Figure 3). Comparing diagnostic performance of RT-PCR and the antigen test in paired NPS and saliva samples that tested positive in symptomatics (N = 77) and asymptomatics (N = 6), we observed the expected lower sensitivity of the antigen test in samples with lower viral load (higher Ct value) irrespective of disease pattern and duration (Figure 3).

Discussion
In the present study, we evaluated the suitability of saliva for detection of SARS-CoV-2 with the Elecsys Anti-SARS-CoV-2 Antigen assay. The combination of a specimen that can be easily and safely obtained with an accurate and rapid detection method would expand the range of strategies for diagnosing early SARS-CoV-2 infection. NPS is considered the optimal material for detection of SARS-CoV-2 in early infection [1]. However, its collection requires trained personnel, which are exposed to a potential risk
The Elecsys Anti-SARS-CoV-2 Antigen assay allows for a rapid detection of the SARS-CoV-2 nucleocapsid protein in NPS and oropharyngeal swab samples with a good sensitivity for symptomatic patients. Our study confirms the high agreement between the Elecsys Anti-SARS-CoV-2 Antigen assay and the cobas SARS-CoV-2 IVD test for NPS samples as reported by the manufacturer. Notably, in our study cohort, the PPA (86.2% for all NPS samples, N = 187) was even higher than the relative sensitivity reported by the manufacturer (PPA of 60.5%, N = 390). This was also evident from the observed thresholds of detection, which were for a PPA of 100% at Ct ≤ 28 in our cohort compared to Ct < 26 reported by the manufacturer.
In contrast to the NPS samples, the Elecsys Anti-SARS-CoV-2 Antigen assay performed much less well with saliva. For the saliva samples, the PPA (with RT-PCR in saliva) was overall only 40.2%, and SARS-CoV-2 could be detected reliably only in samples with Ct values ≤ 26. It is important to note that saliva collection was not tailored for the use with this specific assay but was based on a method that we optimized for detection of SARS-CoV-2 by RT-PCR and entails a dilution of saliva upon collection. The ability to use the same material for different test systems is preferable in diagnostic laboratories, because it provides flexibility to choose different test systems for reanalysis.
Our study shows that lower sensitivity can occur in antigen tests using saliva as a medium and highlights the need for developers to establish appropriate protocols for saliva testing. It must be considered that a reduced sensitivity of SARS-CoV-2 antigen testing in saliva is likely not restricted to the assay evaluated here, as several studies noted lower sensitivity in saliva [11][12][13][14][15]20,22,[36][37][38][39][40][41][42][43][44][45]. Alternate saliva sampling strategies that allow concentrated saliva processing may thus be of advantage in combination with antigen tests to overcome the sensitivity restrictions. Nevertheless, processing undiluted saliva is challenging due to its viscosity and may not be applicable for automated processing. Optimization of saliva pretreatment will be most effective if tailored by the manufacturers to the respective assay system to avoid interference with detection reagents.
Our results show a clear advantage of NPS as a specimen in combination with the probed antigen test. This is in concordance with other antigen test systems which, in general, have shown lower sensitivity compared to NPS [35,[42][43][44][45][46][47][48][49][50][51][52][53][54][55][56]. One of the automated antigen tests previously evaluated for saliva samples is the Lumipulse G SARS-CoV-2 Ag assay, a chemiluminescent enzyme immunoassay which received the CE marking for qualitative and quantitative detection of the SARS-CoV-2 N antigen on both saliva and NPS samples. Sensitivities relative to the corresponding molecular reference test ranged from 41.3% to 88.9%; specificities were reported in four of the five studies and ranged from 96.9% to 98.6% [42][43][44]46,51]. It has to be noted, however, that in the study reporting the highest sensitivity (i.e., 88.9%), only nine RT-PCR-positive saliva samples were evaluated. Another automated antigen test evaluated for saliva is the Simoa SARS-CoV-2 N Protein Antigen Test, which is an EUA for the qualitative detection of the N antigen from SARS-CoV-2 in NPS specimens. PPA and NPA of this assay with molecular testing were 92.3% and 98.1% (N = 26) for days 1-7 [53]. Thus, compared to these two SARS-CoV-2 antigen assays, the Elecsys Anti-SARS-CoV-2 Antigen assay has a lower sensitivity but higher specificity. The sensitivity comparison must, however, be treated with caution, as sensitivity will vary depending on the virus load (Ct) of the included test samples.
The current study was designed as proof of principle analysis to define the capacity of the Elecsys antigen tests in detecting SARS-CoV-2 in saliva. We based our analysis on comparing efficacy of the test in NPS and saliva with a known viral load. We ascertained a representative range of viral loads in the test cohort samples. The study was not designed to assess the performance of the antigen test in saliva of specific patient groups (children versus adults) or disease stages (asymptomatic versus symptomatic). While extended analyses on the use of saliva antigen testing in different disease stages would have been of interest, the low sensitivity we observed in saliva rendered these study extensions unfortunately unnecessary.
The lower sensitivity in saliva with the antigen test probed here highlights the general importance of carefully evaluating saliva as a test material. As we showed, conducting head-to-head comparisons of NPS and saliva for both PCR and antigen test is critical to dissect if matrix effects in saliva exist. As only few studies have assessed saliva in the context of SARS-CoV-2 antigen tests, and those that have, recorded variable results [35,[41][42][43][44][45][46][47][48][49][50][51][52][53][54][55][56], it can currently not be excluded that antigen tests generally show a reduced sensitivity across diverse tests systems. Alternatively, these effects may be test-dependent and can be modest enough to allow application in mass testing as a recent study suggests [41].
In sum, there is a general agreement based on published studies and the data presented here that SARS-CoV-2 antigen tests in saliva are less sensitive and, thus, should only be considered when a clear benefit for clinical diagnosis can be established.
Supplementary Materials: The following are available online at https://www.mdpi.com/article/10 .3390/microorganisms9081700/s1, Table S1: Raw data and donor demographics. Funding: This work was supported by the Swiss federal office of public health (FOPH) and the University of Zurich Foundation to A.T. Roche Diagnostics supported a prior study with PCR kits and consumables for the paired NPS and saliva analyses, data of which are shown in this study. Roche Diagnostics supported the present study with Elecsys SARS-CoV-2 Antigen test kits. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
Institutional Review Board Statement: Ethical review and approval were waived for this study by the Zurich Cantonal Ethics Commission as specimen were leftover from routine diagnostics and a previous study [21] approved with an ethics waiver by the Zurich Cantonal Ethics Commission (Req-2020-00398).

Informed Consent Statement:
No patient consent was required as ethical review and approval were waived for this study by the Zurich Cantonal Ethics Commission (Req-2020-00398).
Data Availability Statement: All raw data are provided as Supplementary Materials.