Accuracy of diagnostic tests for respiratory syncytial virus infection within a paediatric hospital population in Kilifi County, Kenya

Background: Respiratory syncytial virus (RSV)-induced lower respiratory tract disease is a prominent cause of hospitalisation among children aged <5 years in developing countries. Accurate and rapid diagnostic tests are central to informing effective patient management and surveillance efforts geared towards quantifying RSV disease burden. This study sought to estimate the sensitivity (Se), specificity (Sp) (along with the associated factors) and predictive values of a direct immunofluorescence test (IFAT), and two real-time reverse transcription polymerase chain reaction (rRT-PCR) assays for RSV infection within a paediatric hospital population: a multiplex rRT-PCR (MPX) and Fast-Track Diagnostics ® (FTD) Respiratory Pathogens 33 (Resp-33) rRT-PCR. Methods: The study enlisted 1458 paediatrics aged ≤59 months admitted with acute respiratory illness at the Kilifi County Hospital between August 2011 and December 2013. A Bayesian latent class modelling framework was employed to infer the tests’ estimates based on the patients’ diagnostic data from the three tests. Results: The tests posted statistically similar Se estimates: IFAT (93.7%, [90.7; 95.0]), FTD (97.8%, [94.6; 99.4]) and MPX (97.5%, [94.2; 99.3]). As for Sp, FTD registered a lower estimate (97.4%, [96.2; 98.2]) than MPX (99.7%, [99.0; 100.0]) but similar to IFAT (99.0%, [98.2; 99.6]). The negative and positive predictive values were strong (>91%) and closely mimicked the pattern given by the Se and Sp values respectively. None of the examined covariates (age, sex and pneumonia status) significantly influenced the accuracy of the tests. Conclusions: The evaluation found little to choose between the three diagnostic tests. Nonetheless, with its relative affordability, the conventional IFAT continues to hold promise for use in patient care and surveillance activities for RSV infection within settings where children are hospitalised with severe acute respiratory illness.


Introduction
Respiratory syncytial virus (RSV) is a leading cause of bronchiolitis and pneumonia globally, with an estimated 3.2 million hospital admissions and 59,600 in-hospital deaths being registered in children aged <5 years in 2015 1 . In Kilifi, Kenya's coast, RSV is associated with seasonal epidemics characterised by high under-five hospitalisation rates 2,3 .
As RSV clinical presentation is indistinguishable from other acute respiratory infections, virologic testing of respiratory secretions is necessary to establish an RSV diagnosis 4,5 . Since 2002, longitudinal surveillance for RSV at the Kilifi County Hospital (KCH), seeking to strengthen the understanding of paediatric RSV epidemiology, has depended on a direct immunofluorescence test (IFAT) (RSV DFA kit, light Diagnostics TM ) as the standard diagnostic tool 2,6 . Of note, IFAT's merits stem from its good performance (high sensitivity (Se) and specificity (Sp)) when used during peak RSV season, ease of use and interpretation, rapidity and affordability 7 . Nevertheless, the test's accuracy may vary depending on the virus strain targeted and sufficiency of the specimen 8-10 . Furthermore, the test's Se may suffer especially in immunocompromised individuals and during RSV inter-epidemic periods when virus transmission is low 11,12 .
To enhance the detection of RSV, as of 2008, RSV surveillance at KCH has integrated the use of real-time reverse transcription polymerase chain reaction (rRT-PCR) assays, namely: a custom (in-house) multiplex rRT-PCR (MPX) and Fast-Track Diagnostics ® (FTD) Respiratory Pathogens 33 (Resp-33) rRT-PCR 6 . PCR assays possess superior Se to immunofluorescence tests, particularly during off-seasons, owing to their comparably lower viral detection limits 13 . They afford further advantages in permitting the quantification of viral loads in samples and the detection of multiple respiratory viruses. However, owing to the tests' costliness, their applicability in most resource-limited settings is restricted. Moreover, since viral particles may be detectable by the tests in healthy subjects, thereby compromising their Sp, the clinical usefulness of PCRs remains vague 14 . An understanding of the performance of these RSV diagnostics in a clinical setting is valuable for informing clinical decisionmaking and supporting RSV-targeted genetic studies.
Conventionally, evaluation of the performance of RSV diagnostics has relied on the use of imperfect reference tests 15-18 that may introduce bias to the accuracy estimates of index tests. Further, a call for alternative evaluation approaches is justified especially when interest lies in demonstrating the superiority of a novel tests over an existing reference standard 19 . Bayesian latent class models (BLCMs) permit the quantification of test accuracy without knowledge on the true infection status of subjects 20 .
When data are available on covariate factors believed to affect the characteristics of tests under evaluation, stratum-specific estimates are derivable and, arguably, afford greater utility than pooled estimates in inhomogeneous populations 21 . Notably, Se and Sp are properties specific to a test; unlike predictive values, they do not provide information as to the probability that an individual has a certain disease considering their test status. Thus, the objective of the present study was to derive the Se and Sp together with the predictive values of IFAT, MPX and FTD tests for RSV infection within a paediatric hospital population, while assessing the effect of specific factors on the accuracy of the tests.

Study setting and population
As part of the existing long-term surveillance for RSV disease in children at KCH 2 , the study enrolled paediatric patients aged ≤59 months admitted with syndromic pneumonia (as described by Hammitt et al. 22 ) between August 2011 and December 2013. Notably, KCH is the primary referral facility in Kilifi County admitting ~4000 paediatric patients annually.
Here, for children aged <5 years, hospitalisation rates for RSV have been observed in the region of 271-317 admissions per 100,000 children annually 2 . Additionally, RSV epidemics in Kilifi are strongly seasonal usually occurring between the months of November of one year and May of the next 2,23 .

Sample collection
On admission, paired nasopharyngeal and oropharyngeal samples were collected and processed as previously described 24 . The samples were stored in cooler boxes and delivered to the hospital's microbiology laboratory for processing within 2 hours. At the lab, the samples were stored at 4°C until processedusually within 48 hours of collection. The specimens were screened for the presence of RSV by IFAT, MPX and FTD tests. Aside from the samples, the patients' sociodemographic characteristics (date of admission, sex, age, area of residence and presenting symptoms) were recorded.

Ethical considerations
Prior to enlisting the paediatric patients into the study, written informed consent was obtained from their parents/guardians. Approval for the study was granted by the Kenya Medical Research Institute (KEMRI) Ethical Review Committee (SERU No. 3178 and SCC Nos. 1858 and 1526) and Oxford Tropical Ethical Review Committee, UK (60-90).

Target condition
The infection (latent) status targeted by the three tests (IFAT, MPX and FTD) constitutes an RSV-laden respiratory sample carrying either the live virus or its antigens at any concentration.

IFAT
A Light Diagnostics TM RSV DFA kit (Cat No. 3125, Chemicon, USA) was used to test for the presence of RSV in respiratory samples. The IFAT procedure was run as per manufacturer's instructions. Briefly, 200 µl of specimen was used for slide preparation following a centrifugation process (Cytospin 3, Themo Shandon) at 1800 rpm for 10 min. A drop of Light Diagnostics RSV DFA reagent was added to the fixed cell preparation. The ready slide was then incubated in a moist chamber at 37°C for 30 min. Excess reagent was washed off in a fresh change of phosphate buffered saline, with the slide allowed to air dry at room temperature. After adding a drop of mounting fluid to the centre of each well, the stained specimen was examined with an epifluorescence microscope at ×200-500 magnification. An observation of apple-green fluorescence in one or more cells of the stained specimen constituted a positive result.

MPX assay
The MPX assay has been detailed elsewhere 24 . Briefly, 140 µl of respiratory specimen was employed to extract RNA by Qiacube HT using an RNeasy extraction kit (Qiagen, Germany).
The extracted RNA was subsequently tested for RSV (A and B) by a multiplex real-time PCR assay in triplex sets (Qiagen, ABI 7500 system). Samples yielding cycle threshold (Ct) values ≤35 were deemed positive for RSV.

FTD assay
The procedure for the FTD assay is as previously described 25 . Briefly, from a 400µl respiratory specimen, RNA was extracted using the NucliSENS easyMAG platform (bioMérieux, France). The extracted RNA was screened for RSV using the FTD Resp-33 PCR Kit. Quantitative PCR data were generated by creation of standard curves using serial dilutions of plasmid standards provided by FTD, with pathogen density being ascertained from the sample Ct values. As with the MPX assay, Ct values ≤35 were considered positive for RSV.

Population classification
The KCH is nested within the Kilifi Health and Demographic Surveillance System (KHDSS) area in coastal Kenya -stretching about 891 km 2 and spanning a resident population of approximately 260,000 persons. Of note, KHDSS was established in 2000 with a view to monitoring vital statistics as well as, among other goals, quantifying the magnitude of significant local childhood ailments 26 . Consequently, the diagnostic data represented a sample drawn from this single target population that informed the estimation of the Se and Sp of the tests.

Statistical analysis
A Bayesian modelling framework was employed to derive the prevalence, Se and Sp estimates along with the predictive values of the tests. The BLCM was fitted in OpenBUGS software (v 3.2.2) 27 but called from R (v. 3.6.2) via the 'BRugs' package (v0.9-0) 28 . Importantly, the model design and reporting were guided by the standards for the reporting of diagnostic accuracy studies that use BLCMs (STARD-BLCM) 29 . The Bayesian code is available as underlying data 30 .
Essentially, BLCMs are premised on three key assumptions: (i) the target population should consist of two or more subpopulations with different prevalences, (ii) the Se and Sp of the index tests should be conditionally independent given the disease status and (iii) the Se and Sp of the tests should remain constant across the subpopulations 31 . By definition, conditional independence implies that given a subject's disease status, knowledge of results of one test does not alter the expectation about the result of a subsequent test 32 . In our context, given an individual's RSV infection status, the IFAT test was presumed to be conditionally independent to the two PCR assays considering the tests' distinct targets: antigens versus genes. However, to allow for dependence between the two PCRs, we specified two conditional covariances between pairs of the Se and Sp (γ Se , γ Sp ) of the assays as detailed by Gardner et al. 33 . Values of γ Se or γ Sp excluding zero denote significant dependence in Se or Sp estimates of the tests. To assess the separate effects of 'sex', 'age' (categorised into <6 months and ≥6 months 34 ) and 'pneumonia status' on the Se and Sp of the tests, the assumption on constancy of the test characteristics was relaxed by stratifying the single population into subpopulations specified by the covariate levels. As for pneumonia, a child's presenting symptoms following admission were used to infer their pneumonia status (severe or very severe) as previously defined 22 . The stratification permitted the estimation of stratum-specific tests estimates. Differences between the stratified estimates were tested using a Bayesian P-value. Two Markov Chain Monte Carlo chains were used to initiate the models. For each chain, a total of 70,000 iterations were run with the initial 20,000 discarded as the burn-in phase. Convergence of the chains was assessed by visual inspection of the time series plots of selected variables and the Gelman-Rubin diagnostic plots. The posterior distribution of the Se and Sp test estimates, subpopulation prevalences together with the predictive values were recorded as the median and associated 95% posterior credible intervals (PCI).

Results
The cross-classified counts of the three-test results by covariate level are presented in With observed disparities in nasopharyngeal RSV loads between ages 42 , sexes 43 and disease severity classes 44,45 , it is plausible that performance of the RSV diagnostics may be affected by these covariates. This evaluation was conducted, but, as depicted in Table 2, the Se and Sp estimates of the three tests remained comparable across the examined covariate levels. This observation implies that the accuracy of any of the tests is unaffected by the age, sex and severity status of the presenting paediatric patient. Nevertheless, with advancing age (>5 years), owing to diminished viral titres in nasal secretions, the Se of most antigen-based assays (e.g. IFAT) may be undermined 7 , thus precluding their utility in RSV diagnosis in the aforementioned population.
Of the sample of paediatric patients admitted with acute respiratory illness, 22.2% had an RSV infection -asserting the pathogen's preponderance amongst pneumonic patients in the study setting 23 . Correspondingly, the three tests recorded strong and similar Se estimates; although molecular assays, particularly for RNA viruses, may be affected by primer/probe mismatches which may impair their Se 6 . The FTD assay revealed a lower Sp than MPX but similar to the IFAT test. This  discordance in the Sp results between the two assays may be reflective of the non-Sp of the RSV matrix gene targeted by FTD as compared to MPX's nucleoprotein gene target that permits the latter to effectively discriminate between RSV A and B viruses 6 .
All the tests yielded high NPV and PPV values. As for PPVs, the FTD estimate was lower than MPX but comparable to IFAT -a pattern explained by the tests' specificities. The strong confidence in both negative and positive test outcomes realised from these high predictive values is especially pivotal in patient management so as to: (1) establish the diagnosis of viral as opposed to bacterial cause of respiratory illness hence lessen unnecessary antibiotic prescriptions 46 , (2) minimise supplementary testing such as blood tests and chest X-rays 47 , (3) support cohorting of children to limit nosocomial spread within facilities 48 and (4) shorten the duration of hospital stay 49 . Despite the observed good performance of the PCR diagnostics for RSV, their applicability in many resource-limited settings (outside research) is constrained attributable to their high costs and requirement of well-trained personnel. Accordingly, granted IFAT's similar performance (to the PCR assays) coupled with its relative affordability, these attributes lend the test suitable for long-term use in resource-poor settings to foster patient care and surveillance activities aimed at quantifying RSV burden.
A noteworthy caution is that the estimates derived in the present study relate strictly to severely ill hospitalised children. Since patients presenting to outpatient facilities with less severe illness may, conceivably, harbour lower viral titres in respiratory secretions, generalisability of our findings to this population is not guaranteed.

Conclusions
We

Henry Mwambi
School of Mathematics, Statistics and Computer Science, University of KwaZulu Natal, Pietermaritzburg, South Africa The paper is well written and easy to read yet applying a novel statistical approach namely Bayesian Latent Class Models. The paper employs Bayesian estimation to calculate posterior estimates for diagnostic test measures: sensitivity, specificity, positive and negative predictive values. A detailed comparison of the immunofluorescence test (IFAT), and two assays namely MPX and FTD assay was carried accounting for the effect of three covariate namely age, sex and disease severity. The statistical analysis is well articulated and results clearly reported and tabulated. A little bit more discussion on how the latency comes bout in the model description may be more helpful even to a statistically inclined reader although it is understood this is not a statistical methodology paper. Also, the authors should give a clear statement of why a frequentist approach to the problem is not suitable.

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Partly

Are the conclusions drawn adequately supported by the results?
Yes