SARS-CoV-2 antigen tests for screening of healthcare workers; experience with over 48,000 combined antigen tests and RT-PCR tests

Background To prevent spread to patients and co-workers, health care workers (HCWs) infected with Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) should quickly be identified. Although real time polymerase chain reaction (RT-PCR) is the gold standard, this test takes several hours, during which a HCW is unable to work. Antigen (Ag) tests may be an efficacious means of screening HCWs since they are easy to perform and provide fast results. Methods In this study, 48,010 paired results of Ag-testing and RT-PCR, performed on HCWs between January 2021 and April 2022, were evaluated to determine the diagnostic accuracy of SARS-CoV-2 Ag-tests in diagnosing potentially infectious individuals. This analysis was performed with cycling threshold values (Ct-values) ≤30 and ≤25 as cut-offs. Results Respectively 3.1% (n = 1507) and 0.3% (n = 140) of Ag-tests were positive or indeterminate, and thus indicative for SARS-CoV-2 infection. In total, 2479 (5.2%) RT-PCRs were positive, of which 1529 (61.7%) had a Ct-value ≤25 and 402 (16.2%) a Ct-value between 26 and 30. At Ct-value ≤30 as a cut-off, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of Ag-tests were 79.0%, 99.8%, 93.8% and 99.1%, respectively. At Ct-value ≤25, sensitivity further improved to 92.0%, by which the NPV increased to 99.7%. Conclusions To prevent transmission from HCWs to patients and co-workers, while maintaining workforce capacity, Ag-tests are a valuable addition to RT-PCR tests, as they have a quick turnaround time and excellent sensitivity for identifying individuals with high potential for transmission.

With respect to infection control, it is advised that an infected person completes an isolation period to prevent spreading of the virus to others [1,2,5,6]. Especially in a medical/hospital setting, identifying infectious health care workers (HCWs) is important to prevent spread amongst co-workers and patients. Thus, fast and reliable diagnostic methods are necessary.
The most used technique in the medical setting for SARS-CoV-2 diagnostics is real-time polymerase chain reaction (RT-PCR) [7]. RT-PCR is relatively expensive, time consuming and not universally available. SARS-CoV-2 antigen (Ag) tests are an interesting diagnostic method to screen HCWs since they are relatively cheap, easily manually performed, and provide fast results, potentially allowing the identification of infectious individuals within minutes [8][9][10][11]. Several studies showed that SARS-CoV-2 Ag-tests have high specificity but varying sensitivity [8][9][10][11].
Here, we aimed to determine the value of SARS-CoV-2 Ag-tests in HCW-screening.. Therefore, 48,010 paired results of Ag-testing and RT-PCR, performed on HCWs between January 2021 and April 2022, were evaluated to determine the diagnostic accuracy of SARS-CoV-2 Ag-tests in diagnosing potentially infectious individuals.

Study design
In the University Medical Center Groningen (UMCG), a large tertiary referring center employing >14,000 HCWs, HCWs at risk for a SARS-CoV-2 infection (i.e. having signs/symptoms of a SARS-CoV-2 infection, having a recent close contact with a person positively tested for SARS-CoV-2, traveled to a region/country with a high infection rate [6]) were tested for SARS-CoV-2 with RT-PCR. HCWs planning to work in the UMCG (instead of at home) were also tested with a SARS-CoV-2 Ag-test. Following a negative Ag-test result, HCWs were allowed to work in the hospital with proper personal protection equipment (PPE) awaiting the RT-PCR result. The RT-PCR result would follow within eight hours after sample collection. All HCWs of the UMCG, also including non-medical but essential staff, were included in this testing regimen, which is summarized in Fig. 1.

Sample collection and testing
Respiratory samples were collected by nasopharyngeal swabbing followed by oropharyngeal swabbing, using a nasopharyngeal flocked swab. Swabbing was performed by trained employees. This procedure was done twice. One sample was immediately processed for the SARS-CoV-2 Ag-test. A second sample was collected in 3 mL Universal Transport Medium (Copan, USA) and stored at room temperature until performance of the RT-PCR.
From January-October 2021 and January-April 2022, the Panbio SARS-CoV-2 Ag-test (COVID-19 Ag Rapid Test Device, Abbott, USA) was used. From October 2021 until January 2022 the Biosynex SARS-CoV-2 Ag-test (COVID-19 Ag BSS, Biosynex, France) was used. Ag-tests were performed according to the manufacturers' protocols with the following adaptions: (1) The nasopharyngeal swab was complemented with an oropharyngeal swab. (2) For the Biosynex Ag-test, five instead of four droplets of extracted sample were used to perform the test. Results were confirmed by a second employee (four eyes principle). Ag-tests were classified as negative (control line (C-line) present, test line (T-line) absent), positive (C-line and T-line present) or indeterminate (C-line present, but the two independent observers disagreed on presence of a Tline, or: C-line absent).
Multiple platforms were used for SARS-CoV-2 RT-PCR, including the Alinity-M system (Abbott, USA) using the Alinity-M SARS-CoV-2 assay or the Alinity-M resp-4-plex assay, the GeneXpert DX 4.4a system (Cepheid, USA) using the Xpert Xpress SARS-CoV-2/Flu/RSV kit, and the BioFire FilmArray Torch system (BioMérieux, France) using the BioFire Respiratory 2.1 plus panel kit. All were performed according to the manufacturers' protocols. Also a lab developed RT-PCR was used based on a previously described RT-PCR [7,12], using the NucliSENS EasyMAG (Biomérieux, France) for nucleic acid extraction and the Applied Biosystems 7500 Real-time PCR system (Applied Biosystems, USA) for RT-PCR.

Definitions and inclusion/exclusion
In the analysis for Ag-test diagnostic accuracy, indeterminate Agtests were considered positive for optimal infection prevention measures. Indeterminate RT-PCR results were excluded. Some HCWs were screened multiple times over the study period. These measurements are included as separate screening points. Only screening points with a paired RT-PCR and Ag-test result were included. Different studies report different cycling threshold values (Ct-values) of the SARS-CoV-2 RT-PCR as cut-off for infectiousness [13][14][15][16]. We included both Ct-values ≤30 and ≤25 as cut-offs to show the performance of Ag-tests at both levels. Negative RT-PCR results are included and positive RT-PCR tests with Ct-values above the cut-off are excluded in these analyses.

Statistical analyses
Normality of data was investigated using histograms with fitted normal curves, QQ-plots, Shapiro-Wilk tests, skewness values and kurtosis values. Non-normally distributed continuous variables are presented as medians with interquartile range (IQR). Categorical variables are presented as n (%). Differences between two groups were analyzed using Mann-Whitney U tests. The diagnostic accuracy (sensitivity, specificity and number of correctly classified cases) of Ag-tests was determined using nonparametric ROC analysis, with RT-PCR as the reference standard. Positive predictive value (PPV) was calculated as follows: true positive results/(true positive + false positive results). Negative predictive value (NPV) was calculated as follows: true negative results/(true negative + false negative results). Specificity, sensitivity, correctly classified, PPV and NPV are presented as percentages. The area under the receiver operating curves (AUC) are represented with 95% confidence intervals (CI). A p-value <0.05 was considered significant. The statistical analysis was performed using Stata version 14 (Stata Corporation, College Station, USA).

Population characteristics
From January 2021 until April 2022, a total of 10,381 individual HCWs, were screened once or multiple times for SARS-CoV-2 infection by Ag-test and RT-PCR ( Fig. 1, Supplemental Table 1). In total, 48,010 combined tests were performed. The majority of tests were performed in female HCWs (n = 7634, 73.5%). The median age of the population was 35.2 years (IQR 26.8-48.9 years). The median number of combined tests per individual was 3 (IQR 1-6 screenings) ( Fig Table 2).

Diagnostic accuracy of SARS-CoV-2 Ag-tests
Next, the overall diagnostic accuracy of SARS-CoV-2 Ag-tests was determined in the entire testing population.. The classification of Ag-test results with RT-PCR as the reference standard is shown in Table 1. In 97.8% of screenings, the Ag-test resulted in a correct classification, and a sensitivity of 62.3% and a specificity of 99.8% was observed ( Table 2).   [2,5,6]. In phase 2 (T = 0 min), the test regime was determined. Asymptomatic or mild-symptomatic individuals planning to go to work in the University Medical Center Groningen (UMCG) (instead of at home) on the day of testing were tested with both a SARS-CoV-2 antigen (Ag) test and a SARS-CoV-2 Real-time polymerase chain reaction (RT-PCR). Otherwise, only SARS-CoV-2 RT-PCR was performed. In phase 3 (T = 15-30 min), based on the Ag-test result, it was decided whether a HCW was allowed to work at location awaiting the RT-PCR result (i.e. when the Ag-test result was negative), or should await the PCR result at home (when the Ag-test result was indeterminate (IND) or positive). In phase 4 (T ≤ 8 h), based on the RT-PCR result, the final decision was made on whether a HCW was allowed to work at location or should go into isolation.

T = 0 min
The PPV was 93.9% and the NPV was 98.0% . In total, 140 Ag-tests were classified as indeterminate. Of these, 77 (55.0%) had a negative, and 63 (45.0%) a positive or indeterminate RT-PCR result.
By comparing the AUCs (Table 2), the diagnostic performance of Agtests during different variants of SARS-CoV-2 was compared. A better performance in detecting Delta variants (P = 0.013) and Omicron variants (P<0.001) was observed as compared to detecting Alpha variants, but was similar during the dominancy of the Delta and Omicron variants (P = 0.192). No significant difference in performance was observed between Biosynex and Panbio Ag-tests (P = 0.213). This was also true when specifically looking at tests performed during the dominancy of

Diagnostic accuracy of SARS-CoV-2 Ag-tests for identification of infectious cases
Next, we determined the diagnostic accuracy of SARS-CoV-2 Ag-tests to identify infectious HCWs, with a Ct-value of ≤30 and ≤25 as cut-offs for infectiousness. The classification of Ag-test results with RT-PCR as the reference standard is shown in Table 1. With a cut-off at Ct-value ≤30, 98.9% of Ag-tests resulted in a correct classification (Table 2), and Ag-test sensitivity and specificity was 79.0% and 99.8%, respectively. With Ct-value ≤25 as a cut-off, 99.5% of antigen tests resulted in a correct classification. Sensitivity was 92.0%, by which the NPV increased to 99.7%. With respectively a Ct-value of ≤30 and ≤25 as cutoffs for infectiousness, 405 (0.9%) out of 47,209 Ag-tests and 122 (0.3%) out of 46,807 Ag-tests were false negative (Table 1).
No significant difference in performance (AUC, Table 2) was observed between Biosynex and Panbio Ag-tests, and this holds true for both Ct-value cut-off ≤30 (P = 0.384) and ≤25 (P = 0.973). Also during Alpha variant dominancy, Delta variant dominancy and Omicron variant dominancy, no significant differences in diagnostic performance were observed, and this holds true both with Ct-value cut-off ≤30 (P ≥ 0.730) and ≤25 (P ≥ 0.069).

Discussion
In this study, we investigated the diagnostic accuracy of SARS-CoV-2 Ag-tests as a screening method for HCWs using 48,010 paired SARS-CoV-2 Ag-tests and RT-PCRs. Ag-tests showed a good diagnostic accuracy to classify HCWs as potentially infectious or not. With lower Ctvalues as cut-off for infectiousness, diagnostic accuracy was higher and this was especially true for sensitivity. Together, Ag-tests show a high sensitivity to identify individuals with a high potential for transmission, with the major advantage of providing fast results.  Table 2 for exact numbers per section.  Amongst the included HCWs, Ag-tests correctly identified 45,177 HCWs without SARS-CoV-2, who consequently were all able to work on the day of testing. Thus, during the study period the loss of 45,177 working days was prevented. Since Ag-tests are relatively cheap (approximately 5 Euros) as compared to the costs of a lost working day, including Ag-tests in the testing of HCWs for SARS-CoV-2 is likely to be highly cost-effective, both benefiting health care continuity and hospital finances.
Unfortunately, also false negative Ag-tests were observed amongst the tested HCWs and thus some HCWs were allowed to enter the work floor whilst they had the potential to be infectious for patients and coworkers. Depending on the Ct-value that is considered the cut-off for infectiousness, in our study the false negative rate of Ag-tests was 0.9% (Ct-value ≤30) or 0.3% (Ct-value ≤25). We believe that this rate is relatively low and thus acceptable, especially since our protocol prescribed that all HCWs had to wear proper PPE awaiting the RT-PCR result, which followed the Ag-test result within eight hours. By this we limited any unwanted spread of SARS-CoV-2 from HCWs with, in hindsight, a false negative Ag-test result.
Previously, several studies reported the diagnostic accuracy of SARS-CoV-2 Ag-tests [8][9][10][11]. Our study included a large amount of Ag-tests and, to our knowledge, this is the largest study till date in which all Ag-tests were confirmed by RT-PCR. The size of the dataset provides the opportunity to draw solid conclusions on SARS-CoV-2 Ag-test performance in the context of HCWs screening. Importantly, we did not observe significant differences in Ag-test performance to detect infectious cases, neither between SARS-CoV-2 variants, nor between the two used types of Ag-tests (Biosynex and Panbio). Moreover, we observed that Ag-test sensitivity was positively associated with viral load, and this was in accordance with previous data [10,17].
Our study suggests that Ag-tests could be considered to test HCWs at risk for SARS-CoV-2 infectiousness. Importantly, we believe that several precautions are important: (1) In our center we used trained employees for sample collection Ag-test performance. This is important for a reliable performance and interpretation of Ag-tests, and especially for the interpretation of 'weak positive' Ag-test results [18]. (2) We used a combined nasopharyngeal and oropharyngeal swab, as with nasopharyngeal swab alone some positive HCWs might be missed. (3) Additional RT-PCR for positive Ag-test results should be considered. During low community prevalence of the virus, RT-PCR can be of value to prevent unnecessary isolation due to false positive results. (4) For early detection of SARS-CoV-2 infection, RT-PCR and/or repetitive Ag-testing should be considered, as in the early phase of SARS-CoV-2 infection viral load might be low resulting in limited Ag-test sensitivity. (5) With indeterminate Ag-test results it is important to either repeat the Ag-test and/or perform additional RT-PCR, to provide a final verdict on HCW infectiousness. Together, we think an optimal SARS-CoV-2 screening regime should entail both Ag-testing and the option for additional RT-PCR. Since amongst patients it is also important to be able to identify patients with higher Ct-values (infected but less/not infectious), we believe SARS-CoV-2 RT-PCR should remain the gold standard for patients.
Besides SARS-CoV-2 Ag-tests and RT-PCR, a wide variety of other SARS-CoV-2 testing modalities are available, all with varying diagnostic performance, turnaround time and costs. Depending on, amongst others, the size of the testing population and the locally available testing and laboratory infrastructure, other rapid detection methods, for example reverse transcriptase loop-mediated isothermal Amplification (RT-LAMP) [19][20][21], can be considered to be part of a SARS-CoV-2 testing regime.
Some limitations should be mentioned. First, no clinical data on signs/symptoms of positive HCWs were available. However, every Agtest was paired with a RT-PCR result with Ct-value, providing data on the SARS-CoV-2 status and degree of infectiousness of HCWs. Second, only HCWs planning to work in the hospital (instead of at home) were also tested with an Ag-test. Since these HCWs were feeling fit enough for work, it is likely that this group had relatively minor or no signs/ symptoms, and thus might have had a relatively low viral load. Possibly, this has affected the diagnostic accuracy as presented in this study, most likely resulting in an underestimation of Ag-test sensitivity, since previous studies showed a lower sensitivity of the Ag-test in asymptomatic individuals [22][23][24].

Conclusion
In conclusion, to prevent transmission from HCWs to patients and coworkers while maintaining workforce capacity, Ag-tests are a valuable addition to RT-PCR tests, as they have a quick turnaround time and excellent sensitivity for identifying individuals with high potential for transmission. By including Ag-tests in a testing regime to screen HCWs who possibly are infectious for SARS-CoV-2, and allowing HCWs with a negative Ag-test result to work awaiting the RT-PCR result, the loss of a large number of working days can be prevented, with a major impact on both health care continuity and hospital finances.

Ethics approval and consent to participate
The performed study was carried out using pseudonymized data. All procedures contributing to the work described comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Consent for publication
Not applicable. test results (black bars) and true positive Ag-test results (pink bars). RT-PCR results were taken as the reference standard for the Ag-tests. Bin Center=1. Bars represent absolute numbers. (B) Sensitivity (red dots) and specificity (green squares) of the Ag-tests at different Ct-value cut-offs as reference standard. Sensitivity and specificity are represented as percentages.