Secondary Analysis on Malaria RDTs in WHO Product Testing from Round 5 to 8 on False Negative Results in Testing Sample

Background with the widespread use of malaria rapid diagnostic tests (RDTs) in clinical practice, they also show their own challenges. Compared with most of the attention focusing on false positive results in WHO product testing, the same important false negative results attract insucient focus. Method data of 129 malaria RDTs in the summary of WHO rounds 5 to 8 product testing was secondary analyzed in 5 aspects including low parasite density, improper RDTs storage, operation and interpretation, P. falciparum (Pf) with pfhrp2/3 genes deletion, inter-lot variation. Results First, the percentage of tests that achieved a panel detection score (PDS) < 80% at a low parasite density was 20-25% for Pf, and substantially higher for P. vivax (Pv). Second, Some Pv- and Pf- detecting products showed an increasing deterioration with increase of storage temperature. Third, approximately 20-27% products’ performances were unsatised in blood safety and instruction quality; and 30-35% test bands of the RDTs for Pf were barely visible, and the ones for Pv returned a higher proportion of weak band intensity. Fourth, different Lots of each test products may produce inconsistent results. Fifth, many malaria RDTs obtained a low PDS (< 50%) and a high false negative rate (> 50%) when testing Pf with deletion of pfhrp2/3 gene. Conclusion the malaria RDTs currently widely used were overestimated and could cause false negative results in practice. The clinicians should be aware of these shortcomings in order to draw more accurate diagnosis and treatment.

performance of commercially available malaria RDTs [3][4][5][6][7][8][9][10]. Unlike false positive malaria rapid diagnostic test (FP-RDT) results, false negative malaria rapid diagnostic test (FN-RDT) results are yet to receive su cient attention [11]. This article therefore secondarily analyzes the summary results of round 5 to 8 WHO product testing of malaria RDTs, and discuss the causes of FN-RDT with the aim to improve the awareness of the correct interpretation of RDT results, thus further reducing the malaria missed diagnosis and misdiagnosis in clinical practice.

Data collection
This is a secondary data analysis of summary results of WHO product testing on malaria RDTs round 1-8. Since the beginning of this program in 2008, total 332 products were evaluated over the eight rounds of testing. In round 5 of year 2013, a requirement was instituted to resubmit products for re-evaluation within 5 years of original testing. When the same products were resubmitted in subsequent rounds of testing, the second set of results replaced those from the earlier round. In round 8, testing against a panel of HRP2-negative P. falciparum (Pf) was introduced [12]. For the current practical signi cance, WHO carried out the summary which presents an overview of the results of malaria RDT testing in rounds 5 to 8. with the aim to understand the current situation of FN-RDT for malaria, we set up a secondary analysis on the summary data.
There are 129 RDT products nally enrolled into the product assessment on malaria rapid diagnostic test performance in this summary [12]. The summarized data was analyzed in ve aspects, including low parasite density, improper RDT storage, operation and interpretation, Pf with pfhrp2/3 genes deletion, inter-lot variation, so as to clarify the occurrence risk of FN results in malaria RDTs currently widely used in the world.

De nition
Panel detection score (PDS) is calculated with the percentages of positive result per lot in the panel of two RDTs at the lower parasite density or that of a single RDT at the higher parasite density, which was developed to re ect both product sensitivity and reproducibility [12]. Positive rate is calculated as the percentage of the RDTs product that turned a positive test result when tested against a Pf or P. vivax (Pv) sample. The false negative rate (FNR) is equal to 100% minus the positive rate.

Data analysis
Values were presented as mean ± SD (continuous variables), which were compared with the use of the variance analysis. The comparison between different lots of same product is analyzed with repeated measures variance analysis. Summary descriptive statistics using frequency and percentage of characteristics documented for malaria RDTs are categorized by different PDS or FNR. Bar and line graphs were used to show the proportion of malaria RDTs testing result in different PDS or FNR, so as to obtain a clear understanding of the risk of malaria RDTs FN results in clinical practice. SPSS 19.0(SPSS, Chicago, Ill) and Microsoft Excel 2016 were used. A 2-sided P value < 0.05 was considered statistically signi cant.

Results
Performance of malaria RDTs on the sample with low parasite density Among the 129 malaria RDT products analyzed in the summary of WHO product testing from round 5 to 8, there were 42 RDTs against Pf. The results showed that the PDS of nearly 12% products was less than 75%, while only 47.6% products displayed the PDS more than 90%. There were 48 RDTs against Pf and Plasmudium species (Pan), more than 20% ones showed the PDS < 75%, only 31.3% showed the PDS ≥ 90%. The results indicates that RDTs against Pf and other plasmodium species are less sensitive than those against Pf only. As for the RDT products to test Pv, 37.5% RDTs against Pf and Pan showed the PDS below 75%, only 31.3% products' PDS ≥ 90%; 24.2% ones against Pf and Pv/Pvom showed the PDS < 75%, the PDS of 45.5% products was equal to and more than 90%. (Fig. 1)

Performance Of Malaria Rdts In Improper Rdt Storage
Thermal stability is a very important aspect of WHO's regular assessment for RDTs. Speci cally, the detection rate of Pf was evaluated after malaria RDTs had been incubated at room temperature baseline (26℃) and room temperature, 35℃, and 45℃ for 60 days. [1,[3][4][5][6][7][8][9]. This paper summarizes the results of the 5-8th round of product assessment [12] as follows. (Fig. 2) The results show that most of RDTs used to detect Pf can detect Pf 100% at both rather high temperatures of 35℃ and 45℃, the performance does not change with increase of temperature. As for RDT products detecting Pf and all plasmodium species and RDTs detecting Pf and Pv/P.ovale (Po) and malariae (Pm), the positivity rate decreases with increase of incubation temperature. In particular, a lot of RDT products that detect all plasmodium species did not achieve a 100% positivity rate at normal temperature and higher temperature. As a result, these products may generate FN results in testing blood sample with plasmodium.

Performance Of Malaria Rdts In Ease-of-use
For the aspect of ease of use, the blood safety and instruction quality were scored 0-3 and 0-2 respectively and put together to a combined score of 0-5 points. The higher the score, the more convenient the application. The results showed that the lowest score for all products is 2 and the highest is 5 (Fig. 3).    Merely 9.5% of Pf only products could obtain a score of 5, and 6.3% for Pf and pan, 6.1% for Pf and Pv/pvom achieved 5 point. On the contrast, a considerable proportion of products got the points less than or equal to 3, the percentage of ones for Pf was 26.2%, and for Pf and pan, Pf and Pv/pvom were 20.9%, 27.2% respectively.
Another aspect of ease of use is the intensity of the test band of the RDTs. 192 products and 133 test band results were included in this analysis. The band intensity graded as 0 (no visible band), 1,2,3 or 4 (1 being the weakest colour intensity and 4 the strongest) ( Table 1, 2). When testing the Pf samples at low parasite density, 9.6 ± 12.0% of the Pf only products give one result with a bend intensity of 0, and 19.4 ± 11.5% give a band intensity of 1, then the average percentage of products with band intensity of 0 and 1 was about 30%. Meanwhile, Pf and Pan product has the highest proportion of band intensity of 0, approximately 35% of which obtain 0 and 1, and the different proportion of the 5 score ranges among different kinds of RDTs was insigni cant statistically. When testing the Pf samples at low parasite density, intensity score of 0 and 1 account for about 90% in all pan bands of products for pf and pan. When testing Pv samples at the same parasite density, the proportion of RDTs for Pf and Pv/Pvom with intensity of 0 was 6.9 ± 8.0%, and that of 1 is 45.1 ± 22.5%. All the above data shows that there were considerable number of malaria RDTs give barely visible results, and having a higher risk of causing false negative results.

Performance Of Malaria Rdts In uenced By Lot-to-lot Variation
Consistency between test lots is calculated from the number of samples that return positive results on both RDTs tested in that lot against parasite-positive samples.
For low density Pf samples, The positive rate of two lots for Pf only products, Pf and Pan, pf and Pv/pvom were 88 ± 13% vs. 89 ± 12%, 84 ± 12% vs. 85 ± 14%, and 89 ± 6% vs. 89 ± 5%, respectively. For low density Pv samples, the positive rate of Pf and Pan, Pf and Pv/Pvom were 80 ± 27% vs. 80 ± 27% and 90 ± 12% vs. 89 ± 12% respectively. These comparisons suggest that a slight difference between two lots of all products in the assessment test, but there is no statistical signi cance. Nevertheless, it still indicated that different lots of one malaria RDTs product is likely to develop inconsistent results. (Fig. 4) Performance of malaria RDTs against the Pf with deletion of the pfhrp2/3 gene Round 8 of WHO product testing of malaria RDTs included the rst comparative data on RDT performance for detection of P. falciparum with pfhrp2/3 gene deletions [10]. All RDTs were assessed against the panel that contained only low-parasite-density samples or samples with antigen concentrations comparable to 200 parasites/µL. Figure 5 shows that the PDS of RDT products detecting HRP2-negative Pf varied considerably. Of the RDT products designed to detect Pf with Pf-LDH alone or in combination, the products of PDS < 50% accounted for 88.9%, ones of PDS ≥ 80% was nill, and the products with FNR ranged 20% − 50% accounted for 66.7% (Fig. 6). The two pan-LDH RDTs maintained high PDS of more than 80%, and the FNR both were < 20%. As for the 19 products designed to test for Pf with HRP2 only but that also had a non-Pf-LDH line, there were nearly 60% of RDTs with PDS < 20%, none with RDS ≥ 80%, and the FNR of all products were than 50%, 42.1% products were ≥ 80%. Therefore, most malaria RDT products had low sensitivity and high FNR against the HRP2-negative panel, even for products designed to detect Pf with pf-LDH and not HRP2.

Discussion
This analysis on 129 malaria RDTs included in the summary of WHO product testing from round 5 to 8 produced the following results: 1, The proportion of tests that achieved a PDS less than 80% at a low parasite density was 20-25% for Pf, is substantially higher for Pv; 2, As for heat stability, some Pv-and Pf-detecting products showed an increasing deterioration with the increase of storage temperature after 60 days; 3, In term of ease-of-use, approximately 20-27% products need to be improved in blood safety and instruction quality; and 30-35% test bands of the RDTs against Pf were barely visible, those against Pv returned a higher proportion than product against Pf. 4, Different Lots of each tested products can give inconsistent results sometimes. 5, For HRP2-negative Pf, most of malaria RDTs obtained a low PDS of less than 50% and a high FNR, which was more than 50% for some products. In summary, the reliability of the currently widely used malaria RDTs is overestimated to some extent, which have actual risk of false negative results.
Low density malaria infection is a common situation among populations in endemic settings and potentially contribute to ongoing malaria transmission. In 2019, Plucinski MM et.al [13] analyzed 207 outpatients samples from Angola and shows that among HRP2 positive patients with negative RDT result, the positive rate of quantitative reverse transcription-PCR (qRT-PCR) was 45% (95% CI: 35-56%), with a median parasitemia of 3.4 parasites/µL (interquartile range: 0.14-4.8). A substantial proportion of malaria patients have low parasite density levels can't be detected by RDT, but were proved with active infection. As showed in this article, the RDTs to test low parasite density sample got PDS with maximum 20% for Pf, and PDS with maximum 37.5% for Pv. The results demonstrate that these RDT products assessed by WHO to test Pf or Pv are rather probable to produce FN results at low parasite density. A target product pro le (TPP) has been developed for the detection of low-density, subclinical malaria infections [14]. If the sensitivity of TPP is con rmed, it will reduce the FNR of the current RDTs. Moreover, the highly sensitive RDT based on immunochromatographic cassette platform has also been developed for the detection of low-density infections, with detection limit of 40-125 pg/ml HRP2 [15]. The accumulating evidences have demonstrated that the highly sensitive RDT exhibits a twofold higher sensitivity than conventional RDTs and microscopy [16,17]. The highly sensitive RDTs may hold the hope to reduce the human reservoir of infection.
Malaria RDTs are based on capturing parasite antigen with antibodies stabilized on a nitrocellulose(organic) strip [18]. The storage of RDTs for prolonged periods in hot or humid conditions may impair the diagnostic ability thus leading to FN results [19]. This secondary analysis found the positive rate of some products at low parasite density was impaired as incubation temperature increased. As a result, the FNR of those tests increased with change of storage temperature. Malaria is usually endemic in tropical regions, and the hot and humid climates is un t for the storage of RDTs. Also, most malaria affected countries are in sub-Saharan Africa and Southeast Asia which are characterized with poor storage infrastructures, thus increasing the occurrence of FN results. Albertini A.et al. recorded temperature and humidity in Burkina Faso, Senegal, Ethiopia and the Philippines over 13 months [18].
The results showed that the storage and transport temperatures of RDTs regularly exceeded 30.0°C, and the recorded maximum humidity degree was above 94% in the four countries. RDTs were usually exposed to temperatures above recommended limits, leading to poor test performance and high FNR. The RDTs should be selected according to expected eld conditions, and the environmental conditions in supply chains in tropical climates should be managed properly.
RDT is considered as an ideal way for parasite-based diagnosis, particularly in remote and resource limited areas, due to its simplicity, consistency, and accuracy. However, some published studies indicate that the simplicity of RDT was deceptive and the tests were highly user-dependent [20][21][22][23][24]. Rennie W. and his colleagues carried out an observing study on remote health workers and villagers in preparing and interpreting RDTs in Philippines and Laos [25]. The results showed that all test steps from the rst (check expiration date) to the last (13th, dispose of test safely) were all performed below hoped standards.
Seidahmed, OM et al [26] assessed end-users' procedural steps for two main types of RDTs in a mesoendemic area of eastern Sudan and found that the speci c errors caused by test design and manufacturer's instructions signi cantly impair the accuracy of the tests. The results of this analysis show that there are 20% to nearly 30% malaria RDTs getting less than and equal to 3 in the combined score of blood safety and instruction quality. In addition, for the RDTs against Pf sample, 30-35% products displayed the Pf test band intensity is 0 and 1, and almost 90% pan test band intensity less than and equal to 1. Similarly, approximately half of the RDTs against Pf and Pv/Pvom showed the band intensity score less than and equal to 1 for wild Pv sample. A study conducted in Lusaka Province, Zambia [27], indicated that the mean percentage of test results interpreted correctly were 54% in the group guided only by manufacturer's instructions, 80% in the group using only the job aid, 93% using the job aid after receiving a 3-hour training. All the results demonstrated that the performance of RDT strongly depend on the users, and improper operation could inevitably lead to falser positive and FN results. Therefore, in order to improve malaria diagnosis with RDTs, the operators require speci c training and better manufacturer's instructions that take into account of local condition.
Malaria RDT consists of complex biological components that are usually provided from different sources is subjected to various conditions during the manufacturing process, which may affect the quality of the nal product [10]. Although this summary indicated that the inter-lot variation of different products test for different kind parasite antigen in 5-8 rounds is insigni cant through analysis of variance of repeated measures data, however, the actual situation indicated the inter-lot variation does occur. Therefore, currently WHO mandates all manufactures follow ISO 13485:2003 certi cate standards [10] and strongly recommends that a sample from each lot of RDTs should be tested prior to marketing to ensure that they meet the appropriate lot-release criteria.
The parasites have no PfHRP2 protein on reason of pfhrp2 gene deletion can't be recognized by the RDT antigens against PfHRP2 [28]. With the absence of PfHRP2 protein, most of the antibodies used in RDTs for the detection of PfHRP2 could also detect the PfHRP3 protein due to structural homology [29].
The parasites with deletion of pfhrp2/3 gene face less competition within the host that they have an increased probability of transmission success. Consequently, the HRP2-negative Pf should be paid more attention in clinical diagnosis. Therefore, a pfhrp2/3-negative panel was included ad hoc in round 8. The results showed that the PDS and positive rates of the non-HRP2 Pf -speci c tests and test lines in the pfhrp2/3-negative panel was signi cantly lower compared with that in phase 2, and then the FNR of these products was considerately increased. At the same time, the WHO report [10] indicated when positive results on the Pf-detecting test bands were obtained against HRP2-negative samples, the band intensities were generally weak, and inter-lot variation against the HRP2-negative panel was much higher than against the phase-2 Pf panel, which actually lead to higher risk of FNR for these malaria RDTs clinically. As a result, it is necessary to explore the RDT kits with high sensitivity against Pf with HRP2 gene deletion. The alternative biomarkers are in urgent need to improve the accuracy of the current malaria RDTs.
Our analysis has certain limitations. First, not all the products evaluated in every round of WHO malaria RDTs testing were included, the 129 products enrolled in this analysis were based on the summary report of WHO product testing round 1-8 [12], which delist the products that have not been resubmitted as required and discontinue to be manufactured. Second, the FN data was not directly counted and analyzed in this article. In fact, we can evaluate FNR of these products through directly analyzing the sensitivity index such as PDS; Third, this secondary analysis does not include all the possible reasons for the FN results of malaria RDTs, such as the Prozone-Like effect [39][40][41], which was not mentioned in these WHO reports. Clinicians, however, should realize these possible causes, and be vigilant in practical work to avoid possible misdiagnosis and missed diagnosis.

Conclusion
In short, more and more RDT products have been developed for the diagnosis of malaria in the world, especially in areas with low resources. RDTs play an important role in malaria elimination. However, as clinicians and epidemic prevention personnel, we must clearly understand the RDTs' shortcomings, and accurately interpret those results, in order to draw more accurate judgments and treatments, maximize the effectiveness of RDT products. Moreover, it is necessary to explore more sensitive and speci c methods for malaria diagnosis. Authors' contributions: Biao Xu and Bo Tu conceived, designed and write the manuscript, and analyzed and interpreted the data, who contributed equally to this article. Fang Chu, Mohamed Jalloh, Jinsong Mu, Junjie Zheng made substantial contributions to analyze the data and modify the manuscript. Weiwei Chen, conceived and designed this research article and revised the manuscript critically. All authors read and approved the nal manuscript. Summary distribution of malaria RDT rounds 5-8 in combined score of blood safety and instruction quality