Diagnostic performance of influenza viruses and RSV rapid antigen detection tests in children in tertiary care

Highlights • The here studied rapid antigen detection test (RADT) BinaxNOW RSV has a high sensitivity and positive predictive value.• RADT BinaxNOW Influenza A&B has a relatively low sensitivity and positive predictive value.• We advise a restricted use of RADT BinaxNOW Influenza A&B in a tertiary paediatric care setting.


Background
Influenza viruses and respiratory syncytial viruses (RSV) cause acute respiratory tract infections (ARTIs) in children, being a leading cause of hospitalization [1][2][3]. Identification of both viruses is important for disease management, as the presence of these infections may require specific treatment (i.e. oseltamivir) and hospital containment measures. The current gold standard for detection of these viruses is real-time reverse transcriptase PCR (RT-PCR) [4]. This is however not performed in all hospitals, as it requires a molecular diagnostic laboratory with specialized personnel and equipment. Instead, rapid antigen detection tests (RADTs) are used as these assays are easier and cheaper to perform and less time-consuming [5][6][7]. The performance of these tests depends on factors like time between disease onset and sampling, quality and type of specimen and epidemiological parameters [8]. Diagnostic value and clinical usefulness of RADTs for influenza diagnosis vary greatly [5][6][7][9][10][11][12]. This prompted us to evaluate the diagnostic performance of the routinely used RADTs (manufactured by Alere BinaxNOW ® ) for these two viruses as used in our tertiary care paediatric hospital.

Objectives
Comparing diagnostic performances of two RADTs, BinaxNow Influenza AB ® (BNI) and BinaxNow RSV ® (BNR), with those of RT-PCR in samples of paediatric patients attending our tertiary care centre with ARTIs for a period of almost eight consecutive years. Discrepant data were subsequently compared with those of virus isolation and direct immunofluorescence (D-IF) assays.

Study design
This study was conducted from November 2005 through September 2013, we identified paediatric patients between 0 and 5 years who attended Erasmus MC-Sophia's emergency department, out-patient-clinic and those who were hospitalized in this period. To analyse the performance of the BNI and BNR compared to RT-PCR we selected 521 nasal washings of 489 patients with a median age of 4 months (minimum 0.03-maximum 58 months, lower interquartile range 1.6-upper interquartile range 9.8) and 55% (268/489) were male. Nasal washings were obtained during routine clinical practice in symptomatic children and were tested immediately after sampling by trained laboratory personnel using all four diagnostic methods. Multiple samples from the same patient were included in our analysis. Therefore patients are referred to as cases. Data regarding gender, age and hospital admission were obtained from the electronic patient files.

Ethics
Data collection and analyses were conducted on anonymized samples, which does not require further medical ethics review as consented by our Medical ethical board (MEC-2015-306).

RT-PCR gold standard
All nasal washings were tested for the presence of selected viruses by means of RT-PCR with primers and probes sets used in the routine setting of our department [13]. In short, RNA and DNA were extracted using MagnaPureLC (Roche Diagnostics, Almere, the Netherlands) and the total nucleic acid isolation kit. The extractions were internally controlled by addition of a known concentration of phocine distemper virus (PDV) and phocine herpes virus (PHV). Uni-plex RT-PCR was used to detect RSV-A, RSV-B, human rhinovirus (HRV), parainfluenza virus (PIV) type 3 (PIV-3), adenovirus (ADV), and human bocavirus (HBoV). Duplex reactions were performed combining influenza A virus and PDV, influenza B virus and human coronavirus (HCoV) OC43 (HCoVOC43), human metapneumovirus (HMPV) and PIV-2, HCoV229E and PIV-4, and HCoVNL63 and PIV-1. A cycle threshold value (Ct-value) of <40 was defined positive for any virus. RT-PCRs were developed in-house for influenza viruses and RSV-A and validated [13]. RSV-B primers and probes were used as reported by Dewhurst-Maridor et al. [14].

Rapid antigen detection tests (RADTs)
Alere BinaxNOW ® Influenza A and B (BNI) and Alere BinaxNOW ® RSV (BNR) (Scarborough, Maine, USA) are commercially available in vitro immunochromatographic assays for the qualitative detection with monoclonal antibodies directed against influenza A and B virus nucleoproteins and RSV fusion protein antigen, respectively. Nasal washings were obtained using standard protocols and rapid antigen testing was performed as described by the manufacturer. For our analyses the test results of BNI influenza A and influenza B were combined into a single influenza BNI dataset as influenza B was not encountered frequently with only four influenza B BNI positive samples, two of which were influenza B RT-PCR positive.

Virus isolation assay
Virus isolation assays were always performed in combination with D-IF. Madin-Darby Canine Kidney (MDCK) cell line (NBL-2) (ATCC ® CCL-34 TM ) and the human cell line HEp-2 (ATCC ® CCL-23 TM ) were used to isolate influenza viruses and RSV respectively. Virus cultures were regularly checked for cytopathic effect by light microscopy. Immunofluorescence with fluorescein isothiocyanate (FITC) labeled monoclonal antibodies was used to confirm the presence of influenza virus or RSV [15].

Direct immunofluorescence (D-IF) assays in clinical specimens
Cells were isolated from nasal washings, dried on microscope slides, and fixed with acetone. Subsequently, cells were stained with FITC conjugated monoclonal antibodies against influenza A virus, influenza B virus or RSV (IMAGEN TM Influenza A and B and IMAGEN TM RSV, Hampshire, United Kingdom). Specimens were incubated with FITC-conjugated antibodies for 15 min at 37 • C, subsequently excess reagent was washed off with phosphate buffered saline. The stained area was then mounted and viewed by fluorescent microscopy.

Comparison between tests
The focus of our study was to compare data obtained with two RADTs, BNI and BNR with those obtained by RT-PCR as gold standard. We defined false-negative tests as those for which the rapid test was negative and the gold standard RT-PCR positive; a falsepositive test result was defined if the rapid test tested positive and the gold standard RT-PCR tested negative. We compared the available Ct-values in all respective categories of samples and analysed whether there was an association between Ct-values and RADTs results and hospitalization. For influenza all Ct-values were available, for RSV Ct-values were available for 183/204 (90%) of the performed tests. Missing Ct-values were from samples tested in 2005 and 2006 when routine input of Ct-values in our laboratory system was not yet performed and digital documentation was not available. Finally, false-negative and false-positive test results were compared to test results obtained with the other virus detection methods: virus isolation and D-IF assays.

Statistical analyses
The main outcomes of this study were the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the BNI and BNR rapid test results compared to RT-PCR during the total study period and during viral season (October 1st through March 31st). Ct-values were compared with Mann-Whitney U tests.   1). We also calculated these parameters only with samples obtained in the period from October 1st through March 31st, when respiratory viruses are more prevalent in the Netherlands. Sensitivity and specificity decreased with 2% and 1% respectively (69%-67% and 96%-95%). PPV and NPV both decreased with 1% from 55% to 54% and 98%-97% respectively.

False-positive rapid antigen detection tests
Of the 20 influenza false-positive cases, six tested positive in BNR (6/20, 30%). Moreover, six cases tested RSV RT-PCR positive (6/20, 30%) of which five were also BNR positive. In eight samples another respiratory virus than influenza virus or RSV was detected with RT-PCR (8/20, 40%). For two samples the D-IF was positive for influenza, in accordance with the BNI, but for both virus isolation did not yield influenza virus (see Supplemental For BNR only five false-positive cases were found. Three of these were RSV positive in virus isolation and D-IF (3/5, 60%). One case tested negative in all methods except for BNI (see Supplemental  Table 2b in the online version at doi: 10.1016/j.jcv.2016.03.022). The sensitivity (80%), specificity (99%) and PPV (99%) increased if we considered the three RSV positive virus isolations as true-positive cases, resulting in only two false-positive cases (2/514, 0,4%).
Comparing the Ct-values of the respective categories, BNI and BNR cases that were false-negative displayed overall higher Ct-values (p = 0.012 vs p < 0.0001) (Fig. 1A and B). However, no differences were found in Ct-values of hospitalized and non-hospitalized patients within the respective case groups (p > 0.5 for both BNI and BNR) ( Fig. 2A and B). The BNI test result did not differentiate for severe disease with three of the false-negative cases admitted to the paediatric intensive care unit (PICU), but also three

Discussion
The present study evaluated the diagnostic performance of BNI and BNR RADTs in a large number of symptomatic paediatric patients between 0 and 5 years attending our tertiary care paediatric hospital during almost eight consecutive years. By testing fresh nasal washings with RT-PCR, we found a relatively low sensitivity and PPV for BNI. The overall test performance of BNR scored higher for all these aspects. Both BNI and BNR false-negative cases displayed a significantly higher Ct-value compared to all RT-PCR positive and true-positive tested cases.
The accuracy of rapid tests is generally less than that of RT-PCR and virus isolation assays. However, RADTs are valuable as a point-of-care test for their ease of use, fast results and laboratory independence [8]. These advantages, and especially the high specificity are important for their use as surveillance tools for influenza outbreaks as recommended by the World Health Organization [8]. Indeed for surveillance purpose a high specificity is of importance, which we found to be the case in our study. However, for clinical management this is not sufficient. In theory, the decision to start antiviral therapy and to refrain from unnecessary further diagnostic testing and antibiotic use may be based on RADT [16][17][18]. In addition, rapid test results may result in more effective isolation and containment measures [6]. For this purpose assay sensitivity is of utmost importance. Therefore, we conclude that the relatively low sensitivity of the BNI in our tertiary care centre is worrisome. Of note, BNR test performance proved to be better compared to the BNI, although false-negative cases were detected. Based on our results we stopped using BNI and will use rapid PCR-based tests for detection of influenza virus and RSV.
We considered the clinical implications of a false-negative and false-positive test result in relation to hospitalization and found no significant differences between these groups and RT-PCR positive and BNI positive tested cases, indicating that the clinical observation is still pivotal in admission decision-making as suggested by current guidelines. In the present study we were not able to study effects of testing on treatment since antiviral medication was not routinely used in our hospital before 2009. Data on isolation and containment measures were not available.
Although the retrospective nature of our study has inherent limitations, we were able to include more than 500 samples of patients over time. Since, our study spanned a considerably longer period of time than previous studies, we were able to test the clinical feasibility of RADTs "in a real life" clinical setting and for different circulating virus subtypes during eight consecutive seasons. Overall, most studies reported a high specificity and high NPV versus a low sensitivity and low PPV for BNI, which is largely in agreement with our results [5][6][7][9][10][11][12]. We did find a relatively high sensitivity of 79% and an even higher PPV (97%) for BNR, which is in agreement with data obtained in other studies [5,19,20].
During the last decade, RT-PCR has become the gold standard for detecting the presence of respiratory viruses [4]. The downside of RT-PCR is the relatively long time (6-24 h) between sample collection and availability of test results [21]. This makes current RT-PCR formats less useful for admission decision-making and calls for faster methods. Indeed, new rapid point-of-care PCRs are being developed and implemented with a shorter turnaround time [22][23][24][25][26][27]. Studies comparing their performances with those of RT-PCR would allow us to judge their potential for clinical decision-making.
In conclusion, we evaluated the performance of the RADTs BNI and BNR in a tertiary care paediatric hospital setting over eight consecutive years. We showed that sensitivity and PPV of BNI were relatively low (69% and 55%), whereas those of BNR were higher (79% and 97%) when compared to the respective gold standard RT-PCR. False-negative samples consistently displayed high Ct-values, although this did not influence whether patients were hospitalized or not. Given the relatively low sensitivity and PPV of the BNI we strongly advocate a restricted use of BNI or similar rapid influenza antigen detection assays in a tertiary paediatric care setting. In contrast, the higher sensitivity and PPV of the BNR rendered this rapid test more useful, albeit still less sensitive than RT-PCR.

Conflict of interest
PF participates in the IRIS trial, sponsored by Hoffmann-La Roche.

Funding
PF, MK and AO received funding from the EU FP7 project PREPARE (602525) for studies on the pathogenesis of viruses in children. PF and AO received funding from VIRGO project NWO the Netherlands (FES0908). AO received funding from N-RENNT DFG Germany.

Ethical approval
Data collection and analyses were conducted on anonymized samples, which does not require further medical ethics review as consented by our Medical ethical board (MEC-2015-306).