Evaluation of lateral flow devices for postmortem rabies diagnosis in animals in the Philippines: a multicenter study

ABSTRACT Expansion of the use of lateral flow devices (LFD) for animal rabies diagnosis can help mitigate the widespread underreporting of rabies. However, this has been hindered by the limited number and small sample size of previous studies. To overcome this limitation, we conducted a multicenter study with a larger sample size to assess the diagnostic accuracy of the ADTEC LFD for postmortem rabies diagnosis in animals. Thirteen governmental animal diagnostic laboratories in the Philippines were involved in this study, and 791 animals suspected of having rabies were tested using both the direct fluorescence antibody test (DFAT) and ADTEC LFD between August 2021 and October 2022. The LFD demonstrated a sensitivity of 96.3% [95% confidence interval (CI): 94.1%–97.9%] and a specificity of 99.7% (95% CI: 98.4%–100%). Notably, false-negative results were more likely to occur in laboratories with lower annual processing volumes of rabies samples in the previous years (adjusted odds ratio 4.97, 95% CI: 1.49–16.53). In this multicenter study, the high sensitivity and specificity of the LFD for the diagnosis of animal rabies, compared to that of the DFAT, was demonstrated, yet concerns regarding false-negative results remain. In areas with limited experience in processing rabies samples, it is essential to provide comprehensive training and careful attention during implementation.

capable of distinguishing positive rabies antigens from non-specific fluorescence.DRIT, which eliminates the need for fluorescent microscopes, has been established as a simpler alternative to DFAT (11,12).However, it still requires a light microscope and a cold chain to store anti-rabies antibodies, and it is not commercially available.Although various RT-PCR methods have been developed, they still require specialized equipment and are difficult to implement in most endemic areas (9,10,13).Lateral flow devices (LFDs) have been developed to rapidly detect rabies virus antigens.LFDs offer notable advantages, such as rapidity, ease of use, and lack of additional equipment.These tests are particularly valuable in situations where the reference test (DFAT) is unavailable.A critical concern associated with the rabies LFD test is its lower sensitivity than the DFAT test, with variations in sensitivity observed among commercially available kits.Certain kits exhibit unacceptably low diagnostic accuracy (8,14,15).The Bionote LFD kit has been extensively evaluated and has exhibited high sensitivity in previous studies (15)(16)(17)(18)(19)(20)(21)(22)(23).ADTEC LFD has been assessed in multiple countries, including the Philippines, Sri Lanka, Bhutan, and Thailand, demonstrating high sensitivity and specificity (8,(24)(25)(26).A study conducted in the Philippines showed high sensitivity of the Bionote kit 95%; [95% confidence interval (CI), 88%-98%] and the ADTEC kit (94%; 95% CI, 87%-97%), while the Elabscience kit displayed 0% sensitivity.Moreover, when brain samples were collected using the simplified sampling method (straw sampling method), a high sensitivity of 94% (95% CI, 84%-99%) was observed, and thus, this can be used in resource-limited settings (16,26).
We conducted a large-scale multicenter study in a rabies-endemic country.Our primary objectives were to evaluate the diagnostic accuracy of LFD for diagnosing animal rabies and to explore the potential factors associated with the discrepancy between LFD and DFAT.This study represents the first large multicenter evaluation of LFD for rabies diagnosis in a rabies-endemic country to provide robust evidence to the field.

Participating laboratories
The study protocol was reviewed by the research review board of the Bureau of Animal Industry in the Philippines.Animal ethical approval was waived because only brain samples from animals with suspected rabies with carcasses or animal heads were submitted to the laboratories for rabies testing.This multicenter study was conducted to assess the diagnostic accuracy of LFD for the postmortem diagnosis of animal rabies, with DFAT serving as the reference test.The study period was between 1 August 2021 and 31 October 2022.In August 2021, 19 government laboratories conducted animal rabies tests using DFAT.In 2018, the postmortem diagnostic testing of rabies using DFAT was conducted on 3,997 animals in the Philippines, of which 1,227 were found to be positive (29).Three laboratories (Research Institute of Tropical Medicine, Regional Animal Laboratory III, and Regional Animal Laboratory III satellite) were excluded because the ADTEC LFD had previously been evaluated in these facilities (8,26).After excluding those 3 laboratories, 16 were invited to participate in the study.Between November 2021 and April 2021, LFD kits were distributed to these 16 laboratories, followed by online training in sample preparation and use of LFDs (8,26).Of the 16 invited laboratories, 13 participated in this study (Fig. 1).The primary reasons for the lack of participation of the three laboratories were the limited number of rabies samples during the study period and their inability to allocate resources due to the occurrence of outbreaks of other infectious diseases, such as African swine fever and avian influenza.We included the Davao City Veterinary Office in the study because the laboratory performs rabies testing in the region instead of the regional animal laboratory IX.

Baseline survey
We initially conducted an online survey to determine the status of rabies testing in participating laboratories from 2019 to 2020 (see Table S1 in the supplemental material).The survey showed that 23-262 samples per laboratory are tested annually for DFAT.Among them, three laboratories processed fewer than 50 samples per year, whereas four laboratories processed more than 200 specimens.Three laboratories had previous experience with LFD tests for rabies.Veterinarians primarily performed DFAT, although non-veterinarians, such as medical technicians, also conducted the tests in some laboratories.

Study procedure
The participating laboratories routinely accept decapitated animal heads and offer DFAT as a complimentary rabies confirmatory test to both government agencies and private individuals.This is implemented as part of a national passive surveillance system (29).In this study, we instructed the laboratories to perform LFD tests for research purposes alongside their routine DFAT.We included the results of this study when the participating laboratories performed both DFAT and LFD tests.The study team created a standardized sample report form to document the characteristics of rabies-suspected animals and the test results.Laboratories were requested to submit a sample report form along with photographs of the LFD results.When rabies-suspected animals were submitted to the laboratory, both LFD and DFAT were performed.This study did not standardize the test order, depending on each laboratory's decision.In this study, laboratories were permitted to perform the LFD test after briefly storing samples in a −20°C freezer if they could not carry out the LFD on the same day as the DFAT.As long-term storage can reduce the sensitivity of LFDs, we excluded samples that had been stored for more than 90 days (14,15,25,30).In this study, blinded comparisons between the two tests were not employed to prevent examiner bias.

Sampling and LFD procedure
The laboratory staff obtained brain tissue samples by either opening the skull or by a simplified sampling method using drinking straws (occipital foramen route brain sampling without opening the skull).The baseline survey revealed that only two laboratories had previously used a simplified sampling method.For this study, we used the Rabies Ag test (ADTEC Co., Ltd., Oita, Japan; lot nos.2102, 2103, 2104, and 2201).Therefore, we recommend using the brainstem as the sample for LFD.The brain tissue was diluted with the assay buffer and homogenized using the sample Masher kits included in the ADTEC LFD kits.The supernatants were added to the sample hole in the cassette and the results were read after 15 min.The final results were observed as red-colored bands in the test and control zones (see Fig. S1) (24).The detailed method ology of the LFD has been described previously (8,26).As we did not standardize the DFAT method for the study, each participating laboratory performed DFAT according to their own manuals.In the manual procedure in the Philippines, it is recommended to use the hippocampus, cerebellum, and brainstem for DFAT (29).Whereas we examined factors contributing to discrepancies between the DFAT and LFD results, we did not conduct additional confirmatory tests, such as RT-PCR, for samples showing discrepan cies.When laboratories received samples, laboratory technicians assessed the sample conditions and defined them as acceptable or unacceptable for the DFAT.For the acceptable condition, no signs of liquefaction or significant degradation are present in the tissues, and they should exhibit a natural pinkish-gray color without any signs of discoloration.For the unacceptable condition, the sample emits a putrid or foul smell, showing liquefaction or any greenish or unnatural discoloration.

User experience
We conducted a user experience survey between 15 July 2022, and 16 January 2023.After the laboratory staff performed a certain number of LFD tests, we requested them to answer online questionnaires and then conducted qualitative interviews online to explore the usability of LFD and gather opinions.

Data management and analysis
The research team extracted data from the report forms and entered them in Google Forms.The data were converted into Microsoft Excel 2019 (Microsoft Corporation, Redmond, WA, USA).Data analysis was performed using the Stata Statistical Software, version 15.1 (StataCorp, College Station, TX, USA).The sensitivity and specificity of LFD, along with 95% CIs, were determined and compared with those of the DFAT, which served as a reference test (29).Categorical variables were used to categorize laboratory groups based on the yearly number of DFAT tests in 2019 and 2020 (≥200, 100-199, and <100).Fisher's exact test was used to identify statistical differences among categorical variables.Statistical significance was set at P < 0.05.We used a multivariate logistic regression model to examine the factors associated with the discrepancy in results and estimate the adjusted odds ratios (AOR).The final model included any variables with a P-value <0.05, as determined by univariate analysis.The reporting of the study followed the guidelines outlined in the STARD (Strengthening the Reporting of Observational Studies in Epidemiology) statement (see Table S2 in the supplemental material).

RESULTS
Between August 2021 and October 2022, 791 results were included in the study.Thirteen results were excluded because of unavailable DFAT results or samples stored for more than 90 days prior to LFD testing.Therefore, 778 results were included in the analysis (Table 1; Table S2 ).Most of the animals were dogs (n = 682, 87.7%), followed by cats (n = 77, 9.9%), and the DFAT positivity was higher for dogs than for cats.Of these animals, 41.8% were aged less than 12 months old (Table 1).The number of test results per laboratory varied from 6 to 172 tests.While three laboratories submitted more than 100 results, six laboratories provided fewer than 30 results.The positivity rates for DFAT in laboratories range from 30% to 70%.Among the LFD tests, 58.2% (n = 453) were performed within 6 days of receiving the samples, whereas 37.8% (n = 294) were performed between 7 and 90 days.A total of 21.1% of the brain specimens were collected by a straw sampling method.Five LFD lots were used in this study.While only four samples were applied to lot no.2012, around 200 kits of other LFD lots were used (nos.2102, 2103, 2104, and 2201).Of the 778 submitted samples, 434 tested positive for DFAT, and 418 tested positive for LFD.The sensitivity and specificity of LFD were 96.3% (95% CI: 94.1%-97.9%)and 99.7% (95% CI: 98.4%-100%), respectively (Table 2).

Discrepancy results
There were 17 discrepancies in the results between the DFAT and LFD, although we did not perform the confirmatory test using other laboratory methods such as RT-PCR.The results of the analysis of the discrepancy between the DFAT and LFD are presented in Table 3.Only one sample showed a negative DFAT result with a positive LFD result, whereas the other samples showed a positive DFAT result with a negative LFD result.LFD lot no.2201 had 12 negative LFD and positive DFAT results.Each laboratory had 0-4 discrepancy results, with significantly higher discrepancy rates observed in laboratories that had performed fewer annual rabies tests in the previous years (P < 0.05).Most specimens were preserved in an acceptable condition.No significant differences were found in the discrepancy rates based on the species, sample storage duration, sample conditions, and brain sampling methods.While a higher discrepancy rate was observed in LFD lot no.2201, it should be noted that more kits from this lot were used in laborato ries with fewer annual rabies tests in previous years.Logistic regression models showed that the AOR for lot no.2201 was not significant (AOR 5.16, 95% CI: 0.56-47.67).However, significantly higher AORs were observed in laboratories with fewer yearly rabies tests in previous years (AOR 4.97, 95% CI: 1.49-16.53).

User experience
The results of the user experience surveys indicate that all users reported LFD to be easier and faster than DFAT (see Table S3).Approximately half of the users perceived the biohazard risk associated with LFD to be lower than that associated with DFAT, whereas  the remaining users considered it to be at the same level.Among users, 85% agreed that the LFD could be used as a screening tool, 69% considered it suitable for routine testing, and 38% regarded it as a confirmatory test.The LFD received high recommendations, particularly in cases where there were multiple human bites from the suspected animal or when the animal exhibited signs of rabies.

DISCUSSION
A multicenter evaluation study conducted in frontline animal diagnostic laboratories in the Philippines demonstrated the high accuracy of the ADTEC LFD, which was supported by a large sample size.Notably, laboratories with low sample processing in previous years exhibited a higher rate of false negative results than other laboratories.
In our literature review, we identified 21 studies that evaluated the diagnostic accuracy of LFD for postmortem rabies diagnosis in animals compared to DFAT (8,(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(30)(31)(32)(33)(34).Among these studies, only 2 had over 200 samples (18,25), while 11 studies had sample sizes ranging from 100 to 200 (8, 14-17, 19, 22, 26, 27, 34).The remaining studies had sample sizes below <100 (20,21,24,26,28,30,31,33). The largest study included 417 samples, although it used stored samples from three different countries (25).Compared to previous studies, our study had a larger sample size, which enhanced the statistical robustness and increased the generalizability of the findings.Furthermore, the sensitivity in our study was either higher or similarly high compared to the sensitivities reported in other studies.Remarkably, a multicenter study conducted by several international reference laboratories reported inadequate sensitivity for five commercially available LFDs (14).These results were in contrast with the findings of other studies conducted in frontline laboratories (8, 17-19, 22, 26).One possible explanation for the lower sensitivity observed in this study is the use of brain specimens with extended storage periods (14,15,25,30).In contrast, studies using fresh or short-storage samples have demonstrated higher sensitivity (8, 15-19, 22, 26, 27).Similar findings were observed in our study, in which the majority of samples were fresh or stored for a short period (8,26).Furthermore, it suggests that there may be variations in the performance of different LFD brands and emphasizes the importance of carefully evaluating and selecting the appropriate LFD for accurate rabies diagnosis.Most of the specimens in this study were in good condition and were well-preserved, making them suitable for the DFAT, although decomposed samples were commonly rejected for rabies testing upon receipt, making their inclusion in our study impossible.The primary contributing factor to the reduced sensitivity of rabies LFD is its lower detection limit compared with DFAT, leading to false-negative results in specimens with low antigen levels (22).Addition ally, improper sample preparation is also a potential contributor to lower sensitivity.Inadequate homogenization of the brain tissue can be a potential cause of false negative results.In our study, we identified 17 discrepancies in results (2.2%) between the DFAT and LFD.These discrepancies were more commonly observed in laboratories that had processed fewer rabies samples in the previous years.Among the 17 discrepant results observed between DFAT and LFD, the majority were false negative results, whereas only one false positive result was observed.Although we noticed a higher occurrence of false negative results associated with LFD lot no.2201 than with other LFD lots, further investigation found that laboratories with fewer rabies samples in previous years used more kits from lot no.2201.Multivariate logistic regression analysis adjusted with the laboratories categorized based on the number of rabies samples processed in previous years showed no significant association between the specific LFD lots and discrepancy in results.Laboratory VI, which processed over 200 samples in the previous year and used three different lots, showed no significant difference in the discrepancy rate between lot 2201 and the other lots (1.89% vs 1.64%, P = 0.91).These findings suggest that laborato ries with limited experience in processing rabies samples may be more prone to false negative results when using LFD.This highlights the importance of proper training and quality assessment when implementing LFD.Additionally, further research is needed to explore the potential factors contributing to the observed discrepancies and to identify strategies to minimize false negative results in LFD testing for rabies diagnosis.
Whereas we agree that the DFAT or DRIT should remain the gold standard for testing, the LFD can serve as a valuable tool.Expanding the availability of LFD testing could have several positive effects, including clarifying the actual risk areas for rabies, facilitating prompt responses to outbreaks, and increasing the overall awareness of the disease.These measures can contribute to strengthening rabies surveillance and its effective rapid control, especially in areas where the reference tests, such as the DFAT, are unavailable.At present, the ADTEC LFD is available in the Philippines only for research purposes, though it is relatively expensive at approximately 1,500 Philippine pesos (25 USD) per test.However, when compared to DFAT or PCR, adopting the LFD-based test for the diagnosis of animal rabies is more straightforward, as it requires neither an initial investment nor additional equipment and is highly effective for rapid diagnoses in remote areas where DFAT is unfeasible.It is essential to assess both the diagnostic accuracy and potential challenges of introducing this method to areas lacking prior experience with rabies testing, as highlighted in this study.
Our study had several limitations.First, we did not standardize the DFAT method ology and quality assessments at the participating laboratories, which limited the verification of DFAT accuracy as the reference test.Second, we were unable to confirm the discrepancy between DFAT and LFD using alternative methods, such as RT-PCR, which could have provided additional validation.The participating laboratories were unable to conduct RT-PCR for rabies, and transporting brain specimens with the highly pathogenic rabies virus to a remote central laboratory was challenging.For subsequent studies, researchers might consider transporting used LFD kits for molecular analysis to overcome this limitation (16,35).Third, a blind assessment of the two tests was not conducted, indicating that the same staff members may have interpreted the results of both DFAT and LFD.This could have introduced bias and potentially affected the agreement between the two tests.Fourth, the variability in the number of test results submitted by the participating laboratories may have influenced the overall findings, particularly because some laboratories had conducted more rabies tests in previous years.This study was not able to evaluate the skills of each technician, such as years of experience.Finally, our study primarily focused on domestic dogs and did not include other wildlife animals.Therefore, the generalizability of our results is limited to endemic areas where domestic dogs are the primary reservoirs.Further research and assessments are necessary to determine the applicability of LFD for the detection of rabies in wild animals.
In conclusion, our study demonstrated the high diagnostic accuracy of LFD for postmortem rabies diagnosis in animals.LFD showed sensitivity and specificity comparable to those of the DFAT reference test.However, certain discrepancies in the results between the LFD and DFAT were observed; these were mainly false negative results.Laboratories with limited experience processing brain specimens may be prone to these discrepancies.Careful training and quality assessment are necessary when introducing LFD.Further research is required to address implementation challenges and optimize LFD utilization in resource-limited settings.

FIG 1
FIG 1 Locations of participating laboratories and sample sizes with direct fluorescent antibody test results.

TABLE 1
Characteristics of samples from animals suspected of having rabies submitted to participating laboratories according to direct fluorescent antibody test results b a Other species include mice, swine, and cattle.b DFAT, direct fluorescent-antibody test; LFD, lateral flow device; ADDRL, animal disease diagnosis and reference laboratory; CAR, cordillera administrative region.

TABLE 2
Sensitivity and specificity of the lateral flow device compared to those of the direct fluorescent antibody test a a DFAT, direct fluorescent-antibody test; LFD, lateral flow device; CI, confidence interval.

TABLE 3
Associations between test characteristics and discrepancy results a