Diagnostic Accuracy of the Abbott BinaxNOW COVID‐19 Antigen Card Test, Puerto Rico

ABSTRACT Background The COVID‐19 pandemic underscored the need for rapid and accurate diagnostic tools. In August 2020, the Abbott BinaxNOW COVID‐19 Antigen Card test became available as a timely and affordable alternative for SARS‐CoV‐2 molecular testing, but its performance may vary due to factors including timing and symptomatology. This study evaluates BinaxNOW diagnostic performance in diverse epidemiological contexts. Methods Using RT‐PCR as reference, we assessed performance of the BinaxNOW COVID‐19 test for SARS‐CoV‐2 detection in anterior nasal swabs from participants of two studies in Puerto Rico from December 2020 to May 2023. Test performance was assessed by days post symptom onset, collection strategy, vaccination status, symptomatology, repeated testing, and RT‐PCR cycle threshold (Ct) values. Results BinaxNOW demonstrated an overall sensitivity of 84.1% and specificity of 98.8%. Sensitivity peaked within 1–6 days after symptom onset (93.2%) and was higher for symptomatic (86.3%) than asymptomatic (67.3%) participants. Sensitivity declined over the course of infection, dropping from 96.3% in the initial test to 48.4% in testing performed 7–14 days later. BinaxNOW showed 99.5% sensitivity in participants with low Ct values (≤ 25) but lower sensitivity (18.2%) for participants with higher Cts (36–40). Conclusions BinaxNOW demonstrated high sensitivity and specificity, particularly in early‐stage infections and symptomatic participants. In situations where test sensitivity is crucial for clinical decision‐making, nucleic acid amplification tests are preferred. These findings highlight the importance of considering clinical and epidemiological context when interpreting test results and emphasize the need for ongoing research to adapt testing strategies to emerging SARS‐CoV‐2 variants.


| Introduction
As of October 2023, the COVID-19 pandemic has led to 771 million confirmed cases of COVID-19 and 7 million deaths globally, with Puerto Rico reporting almost 1.3 million COVID-19 cases and 6000 associated deaths [1].Rapid identification of SARS-CoV-2 infection and subsequent measures to reduce transmission are central to an effective public health response to COVID-19 [2].However, the broad spectrum of clinical manifestations of SARS-CoV-2 infection poses a challenge to the rapid identification of infections and the implementation of effective measures to reduce transmission [2][3][4].Concurrently, the pandemic prompted the development of novel therapies [5][6][7][8] that are designed to shorten COVID-19 symptom duration.Early identification of SARS-CoV-2 infection is crucial for the timely and appropriate administration of therapies, particularly for people at higher risk for severe disease.Many of the novel treatments developed during the pandemic require initiation within a specific window after symptom onset.However, the challenges posed by the broad spectrum of clinical manifestations, including initially asymptomatic and mild cases that can progress to severe disease, make early and accurate detection of SARS-CoV-2 infection essential for effective treatment and prevention strategies.
To identify infected individuals for isolation and appropriate medical therapy, rapid and accurate COVID-19 tests continue to play a crucial role, including those used in clinical and laboratory settings.Although RT-PCR-based testing is frequently available in clinical and laboratory settings for infection detection, its utility can be limited by the expertise required for proper sample management and reporting delays due to the time needed for transport and testing at laboratory facilities [9,10].In many communities, point-of-care rapid antigen tests were deployed to enhance the accessibility and efficiency of SARS-CoV-2 infection detection.Among the available commercial lateral flow antigen tests, the BinaxNOW Antigen Card test has undergone particularly extensive evaluation, demonstrating consistent specificity (> 97%) across multiple cohort studies [11][12][13][14][15].However, sensitivity estimates varied widely in different reports, with potential factors including timing of specimen collection, symptom presence, collection methodology, and viral replication levels, necessitating further validation.
In December 2020, BinaxNOW testing was introduced alongside RT-PCR testing for SARS-CoV-2 in a community cohort and two clinical surveillance sites in Puerto Rico.We evaluated how the performance of BinaxNOW varied by days postonset of symptoms, symptomatology, predominant SARS-CoV-2 variant, vaccination status, collection strategy, repeated tests, and RT-PCR cycle thresholds (Ct).This study leverages its large sample size, including specimens collected at various time points from a unique population in Puerto Rico, to provide a comprehensive evaluation of the BinaxNOW Antigen Card test's performance, contributing to filling an information gap in the use of point-of-care rapid antigen tests for SARS-CoV-2 infection detection.Our findings contribute to a deeper understanding of the test's efficacy and role in augmenting current diagnostic strategies.

| Study Design and Data Collection
The data analyzed is derived from two observational studies in Puerto Rico: the Communities Organized to Prevent Arboviruses (COPA) study and the Sentinel Enhanced Dengue Surveillance System (SEDSS), both of which are conducted by the Ponce Health Sciences University (PHSU) and the US Centers for Disease Control and Prevention's (CDC) Dengue Branch (DB).
COPA is a community-based cohort study established in Ponce, Puerto Rico, in 2018.Study enrollment and data collection activities are described elsewhere [16][17][18].Briefly, study activities include annual interviews and serum collection for arbovirus testing among approximately 3800 participants.Beginning in April 2020, anterior nasal swabs for SARS-CoV-2 RT-PCR testing were collected from participants that reported experiencing COVID-like symptoms (i.e., fever, cough, sore throat, difficulty breathing, diarrhea, body pain, or loss of taste/smell) or within the last 7 days of their annual visits.Additionally, an acute illness surveillance component was initiated via weekly text messages asking participants to report if they or a household member experienced COVID-like symptoms in the past 7 days.Symptomatic participants, as well as those with a prior positive lab test for SARS-CoV-2 in the last 7-21 days and their household contacts, were offered visits for anterior nasal swab collection for SARS-CoV-2 RT-PCR testing.Beginning in December 2020, concurrent collection of a second anterior nasal swab for testing by the BinaxNOW COVID-19 Antigen Card test was offered to all participants with a swab collected for SARS-CoV-2 RT-PCR testing.All nasal swabs were collected by study staff, and BinaxNOW testing was performed within 1 h of collection at the study site.Our analyses include COPA participants who were tested for SARS-CoV-2 between December 2020 and May 2023 using both BinaxNOW and RT-PCR assays.COPA participants may have been tested multiple times in the study period, including during the same and separate illness or exposure events.
Established in May 2012, SEDSS is an active surveillance system that monitors acute febrile and respiratory illnesses in two emergency departments in Ponce, Puerto Rico.In 2018, an additional site was established in an emergency department in San Juan [19][20][21].Patients were eligible for enrollment if they demonstrated fever upon presentation or within the past week (oral temperature ≥ 38°C, axillary temperature ≥ 38.5°C) or cough/dyspnea within the last 14 days (with or without fever).Nasopharyngeal swabs collected at enrollment from participants in SEDSS were tested for SARS-CoV-2 using RT-PCR.Two collection approaches were employed for BinaxNOW testing in one of the two participating emergency departments: staff-collected and participant-collected (self-testing) anterior nasal swabs.Participants underwent staff-collected, self-collected, or both staff-and self-collected anterior nasal swabbing concurrently.Participants were provided with clear and simple instructions for self-collection and testing, including applying drops to the test card, swabbing both nostrils, and following specific steps for test card handling [22].Our analyses included SEDSS participants in the San Juan or Ponce sites tested for SARS-CoV-2 between January and April 2021 using both BinaxNOW and RT-PCR assays.
For both COPA and SEDSS, the RT-PCR assays used included the CDC Real-Time Reverse Transcription PCR Panel for tests performed before December 2021 and the CDC Influenza SARS-CoV-2 (Flu SC2) Multiplex Assay for tests performed December 2021 and later [23,24].

| Statistical Analysis
We reported frequencies of demographic characteristics (age group, sex, ethnicity, race, and Hispanic/Latino), reported chronic medical conditions, COVID-19 vaccine doses, and number of RT-PCR/BinaxNOW tests among all COPA and SEDSS participants with one or more RT-PCR/BinaxNOW test result data available.
Using the SARS-CoV-2 RT-PCR result as our reference standard, we calculated measures of diagnostic accuracy of BinaxNOW tests including sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, and the number needed to diagnose (NND) of BinaxNOW tests compared to RT-PCR tests.Definitions of these measures are given in Table S1.We calculated 95% confidence intervals (CI) for all measures.We used McNemar's test to evaluate differences in proportions of discordant pairs (i.e., the differences between false positives and false negatives) between BinaxNOW and the reference standard, RT-PCR [25].It helps determine if one test is more likely to produce false positives or false negatives compared to the other.To assess discrimination, we calculated the area under the receiver operating characteristic curve (AUC-ROC).AUC-ROC summarizes the trade-off between sensitivity and specificity, where an AUC of 1 indicates perfect discrimination, and 0.5 indicates no discrimination.
For COPA participants with repeated tests, we evaluated BinaxNOW performance for their initial test as well as the repeated test 7-14 days after the initial test.We further stratified this analysis by participant symptom status for the initial and repeated tests.We performed a sensitivity analysis for the repeated tests by restricting to participants who had testing within 6 days of symptom onset to ensure the repeated test was not for a different infection.For repeated tests among COPA participants, tests separated by ≥ 90 days were considered as part of separate illness episodes, and tests within 7-14 days of each other were  considered part of the same illness episode [27].The few COPA participant tests performed between 15 and 89 days of another test were excluded from the analysis.In SEDSS, when both selfcollected and hospital staff-collected swabs were tested, all tests, including the RT-PCR test, were conducted on the same day and included in the analyses.
We fit cubic splines to further understand the relationships between sensitivity and specificity of BinaxNOW by days postonset of symptoms and total number of symptoms.All analyses were done using R software, version 4.3.1 (R Foundation for Statistical Computing, Vienna, Austria).
The sensitivity of BinaxNOW was higher for symptomatic (86.3%) than for asymptomatic (67.3%) participants, whereas specificity estimates were the same (98.8%)for both groups.
Sensitivity did not significantly vary by the number of symptoms reported (Figure S1).For symptomatic participants, one correct diagnosis was obtained for every 1.2 patients tested with BinaxNOW on average during the study period (NND = 1.2, 95% CI: 1.1-1.2) (Table 3).For asymptomatic participants, one correct diagnosis was obtained for every 1.5 patients tested with BinaxNOW on average during the study period (NND = 1.5, 95% CI: 1.3-2.0).The sensitivity and specificity of BinaxNOW showed consistent performance across participants regardless of the number of COVID-19 vaccine doses received, with overlapping confidence intervals for all groups (Table 2).S2).
There were 184 participants who had repeated tests within a single illness or exposure event (7-14 days after the initial test).In the initial test, BinaxNOW demonstrated high sensitivity (96.3%) and specificity (96.0%) for detecting SARS-CoV-2 (Table 2).
During subsequent sample collection and testing 7-14 days later, sensitivity decreased to 48.4%, while specificity remained high at 97.9% (p-value from McNemar's test < 0.001).Restricting to 134 participants who had the initial test within 6 days of symptom onset, the sensitivity was 96.1% for the initial test and 48.8% for the repeated test 7-14 days later.The initial test showed strong overall performance (AUC-ROC = 0.961), whereas the follow-up testing showed a decline in accuracy for identifying positive cases over time (AUC-ROC = 0.731) (Table 3).Sensitivity dropped significantly for participants initially symptomatic (98.7%) and later asymptomatic (23.1%) (Figure 2 and Table S2).
Conversely, sensitivity increased for those initially asymptomatic (50.0%) and later symptomatic (100%), but this difference was not statistically significant possibly due to the limited sample size.
Our findings regarding BinaxNOW test performance in symptomatic and asymptomatic individuals also align with those from other studies [11][12][13][14], showing substantially higher test sensitivity in symptomatic compared to asymptomatic individuals, while maintaining a high level of specificity for both groups.We did not find a clear dose-response relationship between the number of symptoms experienced and sensitivity, but the point estimate for test sensitivity was highest (91.5%) among  participants with 10 or more symptoms.Symptom type and indicators of disease severity, such as low oxygen saturation levels, tachypnea, or requiring hospitalization, rather than simply the number of symptoms reported, may have a greater influence on diagnostic accuracy [31].These results corroborate previous research and highlight the challenges of detecting SARS-CoV-2 infections in asymptomatic cases [11][12][13][14].Clinicians should consider these factors and follow CDC guidelines for using antigen tests, including repeat testing for asymptomatic individuals who were exposed, considering other etiologies for symptomatic individuals, and repeating testing with RT-PCR in situations where sensitivity is of paramount importance according to CDC recommendations [32].FDA recommends repeat testing following a negative result on any at-home COVID-19 antigen test regardless of symptoms to reduce the risk of false negatives [33].
Following infection, SARS-CoV-2 viral replication and shedding precede symptoms, with peak viral titers occurring near the day of symptom onset and declining thereafter [34].This trend is supported by studies indicating that antigen testing demonstrates higher sensitivity early in infection when viral loads are high, while repeated sampling over the illness course correlates with decreasing sensitivity [11,29,30,35,36].Ct values from RT-PCR tests also provide quantity of viral genetic material in the sample (as an approximate proxy for viral load) with increasing Ct values reflecting decreasing viral genetic material [37,38].Our study used the same RT-PCR assay for SEDSS participants, but two different RT-PCR assays were used in COPA, which precludes direct comparison of Ct values due to variation in sensitivity, chemistry of reagents, gene targets, cycle parameters, and others [37].BinaxNOW test showed peak sensitivity (99.5%) when the Ct values of paired RT-PCR tests were 25 or lower, suggesting a higher concentration of viral genetic material, typically indicative of early-stage infection.This is consistent with our findings of reduced sensitivity 7 or more days after symptom onset, as well as those showing a significant decline in sensitivity with repeated testing conducted in samples collected 7-14 days after initial testing.These findings emphasize the importance of testing during the early infection stage and maximizing the utility of isolation and treatment, when indicated.However, BinaxNOW test sensitivity drops significantly (18.2%) for cases with Ct values between 36 and 40, suggesting a diminished capacity to detect positive SARS-CoV-2 cases among individuals with lower viral genetic material concentrations during later stages of infection.
Compared to ancestral variants, Delta and Omicron are characterized by their shorter incubation periods, serial intervals, enhanced immune evasion, and heightened transmissibility [39][40][41].Studies have yielded mixed results in viral load patterns for these variants, with some reporting higher viral loads for Delta [42,43], whereas others report higher viral loads for Omicron BA.1 [44,45].The limited number of tests during the Delta variant dominant period in our study precluded robust comparisons of sensitivity between SARS-CoV-2 variants, and there were overlapping confidence intervals for sensitivity across the variants.One study reported lower BinaxNOW COVID- 19 Antigen test sensitivity for infections with the Omicron variant compared to those with the Delta variant [46], and another found no significant difference in sensitivity between the two variants [47].The impact of infection prevalence, such as the   study, may differ from non-participants regarding healthcareseeking behavior, symptom severity, proximity to healthcare facilities, access to healthcare, socioeconomic factors, and risk perception, potentially introducing selection bias.Our study included pre-Delta, Delta, and Omicron (time period covering BA.1 through XBB.1.5[52]) variants.However, our study population had low SARS-CoV-2 transmission prior to the Omicron variant.More recent Omicron subvariants like EG.5 and FL.1.5.1 may have viral mutations that affect BinaxNOW performance.Furthermore, our study used the dominant variant period as a proxy for the actual variant of the individual, potentially misclassifying cases due to variability within these periods.Lastly, our study focused on a rapid antigen test for SARS-CoV-2 from a single manufacturer.Our findings may not apply to other antigen tests with potentially different performance characteristics.

| Conclusions
Our study provides valuable insights into the diagnostic performance of BinaxNOW COVID-19 Antigen Card Test in different epidemiological contexts.While demonstrating high sensitivity and specificity, our findings highlight the influence of factors such as symptomatology, viral load, and timing of specimen collection on test accuracy.BinaxNOW remains a valuable tool for home use and early infection identification, offering numerous advantages, including low cost, extended shelf life, temperature stability, ease of use, and the ability to identify individuals with high viral loads.However, its application should be considered alongside clinical and epidemiological context [32].Future research should continue to explore the evolving landscape of SARS-CoV-2 variants and the performance of rapid antigen tests across diverse populations to further enhance our understanding and response to COVID-19.

FIGURE 1 |
FIGURE 1 | Sensitivity and specificity of BinaxNOW Antigen test compared to RT-PCR by days postonset of symptoms (N = 1181 paired tests from 921 participants with 0 to 16 days postonset).The blue line represents a cubic spline and grey bands indicate 95% confidence intervals of the model fit.Vertical bars are 95% confidence intervals of BinaxNOW sensitivity and specificity for each days postonset subgroup.There were 1181 tests of both BinaxNOW and RT-PCR.
Abbreviations: COPA, Communities Organized to Prevent Arboviruses; IQR, interquartile range; RT-PCR, reverse transcription polymerase chain reaction; SEDSS, Sentinel Enhanced Dengue Surveillance System.a All repeat testing for SEDSS participants was performed on the same day with one swab collected by a healthcare provider and another self-collected swab.b For repeated tests among COPA participants, tests separated by ≥ 90 days were considered as part of separate illness episodes, and tests within 7-14 days of each other were considered part of the same illness episode.COPA participant tests performed between 15 and 89 days of another test were excluded from the analysis.

TABLE 2
| Comparison of BinaxNOW and RT-PCR (N = 1530 paired a tests from 1207 participants unless stated otherwise).
were 1207 total participants with results from paired BinaxNOW and RT-PCR tests: 943 (78.1%) from COPA and 264 (21.9%) from SEDSS (Table1).The median age of all participants was 36 years (IQR: 17, 49), 57.4% were female, 99.7% were Hispanic/Latino, and 56.3% had reported past diagnosis with one or more chronic medical conditions.Of 799 COPA participants with available COVID-19 vaccine data, 92.5% had received at least two doses, whereas 5.8% remained unvaccinated.All SEDSS participants were unvaccinated and tested before vaccines became widely available in Puerto Rico.Among the 264 SEDSS participants, 58 (22.0%) underwent both staff-collected/ tested and participant-collected/tested BinaxNOW tests, resulting in a total of 322 BinaxNOW tests.In COPA, there were 1208 BinaxNOW tests from the 943 participants from December 2020 to May 2023.Of the 1530 total tests from SEDSS and COPA, 404 (26.4%) were positive for SARS-CoV-2 on the BinaxNOW test and 465 (30.4%) were positive by RT-PCR.

TABLE 2
The sensitivity of BinaxNOW varied significantly depending on the Ct values from positive RT-PCR tests.For Ct values ≤ 25, paired BinaxNOW tests showed 99.5% sensitivity in correctly identifying positive cases (Table4).Conversely, as Ct values increased, test accuracy declined, reaching only 18.2% for Ct values between 36 and 40.

TABLE 3 |
Performance of BinaxNOW compared to RT-PCR (N = 1530 paired a tests from 1207 participants unless stated otherwise).
a There were 1530 tests of both BinaxNOW and RT-PCR.b N = 1192 tests from 923 participants-284 participants were missing symptom onset dates.c N = 1526 tests from 1203 d N = 368 tests from 184 participants.Restricted to participants who had repeated tests.e N = 1359 tests from 1063 participants-144 participants were missing vaccination data.

TABLE 3
This study had several limitations.Our population was composed primarily of individuals that identified as Hispanic/ Latino and Puerto Rican between the ages of 0 and 50 years, which may not fully represent diverse populations or epidemiological conditions found elsewhere.BinaxNOW performance may vary in populations with different demographic characteristics, vaccination rates, or healthcare access.Additionally, participants in our study, comprising individuals seeking medical attention or enrolling in a community-based cohort FIGURE 2 | Sensitivity of BinaxNOW Antigen test compared to RT-PCR for initial tests and repeated tests 7-14 days later by symptom status for the initial and repeated tests (N = 368 paired tests from 184 participants).Additional diagnostic accuracy measures are shown in TableS2.There were 368 tests of both BinaxNOW and RT-PCR.

TABLE 4 |
Sensitivity of BinaxNOW by Ct values of RT-PCR tests (N = 464 paired a tests from 435 participants).This table includes positive RT-PCR tests and corresponding BinaxNOW test results.Of 465 total positive RT-PCR tests, 1 was missing a Ct value; therefore, the sample size includes 464 tests of both BinaxNOW and RT-PCR.
Abbreviation: RT-PCR, reverse transcription polymerase chain reaction.a