Sensitivity and specificity of a rapid point-of-care test for active yaws: a comparative study

Background To eradicate yaws, national control programmes use the Morges strategy (initial mass treatment and biannual resurveys). The resurvey component is designed to actively detect and treat remaining yaws cases and is initiated on the basis of laboratory-supported reactive non-treponemal serology (using the rapid plasma reagin [RPR] test). Unfortunately, the RPR test is available rarely in yaws-endemic areas. We sought to assess a new point-of-care assay—the Dual Path Platform (DPP) syphilis assay, which is based on simultaneous detection of antibodies to treponemal and non-treponemal antigens—for guiding use of antibiotics for yaws eradication. A secondary goal was to ascertain at what timepoint the DPP assay line reverted to negative after treatment.


Introduction
Yaws is a neglected tropical disease caused by Treponema pallidum pertenue and closely related to syphilis. The fi rst sign of infection-a localised papuleis apparent after an incubation period of 10-90 days. Secondary disseminated cutaneous lesions and tertiary destructive lesions of the bones develop with relapsing or persistent infection. 1 According to the WHO simplifi ed pictorial guide, 2 yaws is suspected if multiple papillomas, non-tender ulcers, bone lesions, or plantar hyperkeratosis are present.
Syphilis serological testing has been used in the fi eld to assist with diagnosis of yaws infections. Traditionally, individuals in whom seropositivity for syphilis correlates with clinical signs and symptoms of yaws are presumed to have yaws. Serological diagnosis of yaws requires detection of two distinct antibodies: one against a treponemal antigen and one against a non-treponemal antigen. Non-treponemal agglutination tests (eg, the rapid plasma reagin [RPR] test, and the VDRL slide test) become reactive during the initial stage of infection and generally revert to negative after treatment. [3][4][5] By contrast, treponemal serological tests (eg, the T pallidum haemagglutination assay [TPHA], the T pallidum particle agglutination assay [TPPA], and the fl uorescent treponemal antibody absorption [FTA-Abs] test) remain reactive for life, despite treatment. 6,7 Thus, non-treponemal tests are a better indication of active infection and ongoing transmission in an area. WHO has targeted yaws for eradication by 2020 8 and has developed the Morges strategy, which comprises initial mass treatment followed by surveys every 6 months to actively detect and treat remaining cases. 9,10 Use of one oral dose of azithromycin 11 has made mass treatment simpler, and WHO studies are underway to confi rm effi cacy of this strategy. During retreatment surveys, accurate identifi cation of patients with active infection who need antibiotics is done with non-treponemal tests (ie, the RPR test or VDRL slide test), [3][4][5] but these diagnostic procedures entail use of a sample of serum and must be done in laboratory settings that are rarely available in yaws-endemic areas. Decisions for continued targeted intervention could be made on the basis of commercially available point-of-care treponemal tests. 12,13 Unfortunately, treponemal test results correlate poorly with presence of active infection, particularly after one or more rounds of mass treatment. Therefore, monitoring the eff ect of biannual mass administration of antibiotics with treponemal tests alone is likely to result in unnecessary and costly treatment and could increase the probability of selection for antibiotic-resistant pathogens. 14 A point-of-care immunoassay that simultaneously detects antibodies to non-treponemal and treponemal antigens was developed for diagnosis of syphilis, and the test is expected to work for yaws. [15][16][17] The assay-Dual Path Platform (DPP) Syphilis Screen and Confi rm (Chembio Diagnostic Systems, Medford, NY, USA)-is designed for use in resource-limited settings where challenging conditions (such as lack of electricity, running water, or laboratory equipment) commonly exist. 16 We sought to compare the DPP assay for diagnosis of yaws infection with TPHA and the RPR test as reference standards. A secondary aim of our study was to estimate the changes in DPP optical density values among children with positive non-treponemal and treponemal antibodies 3 and 6 months after treatment.

Participants
We recruited participants from Lihir Island villages in the New Ireland province of Papua New Guinea in April, 2013 (referred to as group 1), and from Karkar Island in the Madang province of Papua New Guinea in October, 2013 (referred to as group 2). We selected these communities because they have a fairly high prevalence of yaws in children. 18,19 We undertook community-based surveys and invited children aged 1-18 years, who were suspected to have yaws by clinical examination, to participate in the study.
We obtained approval for the study from the National Medical Research Advisory Committee of the Papua New Guinea Ministry of Health (MRAC approval 13.25). All participants, or their parents, provided signed informed consent.

Procedures
Two clinicians (PM and RW) with extensive experience in diagnosing yaws examined and classifi ed participants according to the WHO yaws pictorial guide. 2 We took a venous blood sample from every individual and stored it in an EDTA tube for testing with the DPP assay and the standard reference tests. At the end of every clinic day, we transported blood tubes to the local hospital. A laboratory technician separated plasma and froze samples at -20°C until they could be transferred to the Lihir Medical Centre laboratory for further serology testing. On Lihir Island (group 1), a technician did the DPP assay in the Lihir Medical Centre laboratory with the plasma specimens that were frozen before testing. On Karkar Island (group 2), a community health worker did the DPP assay in the fi eld using fi nger-stick whole-blood samples. The two types of specimen-plasma and whole blood-could not be tested for every patient because of diffi culties implementing the study in the rural regions. Laboratory technicians undertook RPR tests and TPHA for both groups at the Lihir Medical Centre laboratory. These technicians randomly selected (using a random number generator and numbered samples) 5% of reactive and non-reactive samples and sent these to an independent microbiology unit (Sullivan Nicolaides Pathology, QLD, Australia), at which TPHA and the RPR test were repeated for quality assurance.
We gave one oral dose of azithromycin (30 mg/kg) to participants with reactive treponemal and non-treponemal serology. We followed up children in group 1 who were dually positive by DPP assay, to monitor changes in optical density 3 and 6 months after treatment.
The DPP Syphilis Screen and Confi rm assay (Chembio Diagnostic Systems) is a point-of-care immunochromatographic device that detects IgG and IgM antibodies to treponemal (T1) and non-treponemal (T2) antigens. 15 It contains two nitrocellulose membrane strips perpendicular to each other that allow independent delivery of the test sample and the detecting protein A and anti-IgM conjugate reagents. The specifi city of the assay for syphilis was established previously against a panel of serum samples obtained from patients with 17 infectious diseases other than syphilis. 15 Furthermore, the point-ofcare assay also identifi ed correctly all 105 serum samples from patients with known stages of syphilis. 15 To do the DPP assay, we used 10 mL of blood or 5 mL of the plasma sample; we adhered strictly to the manufacturer's instructions. One clinician read the results of the DPP assay by naked eye and recorded it in a data collection form. A diff erent clinician, who was unaware of the naked-eye value, inserted the DPP assay cartridge into an automatic reader device (Chembio Diagnostic Systems) that measures numerically the optical density of each test line and stores the results. To ascertain DPP assay positivity, the manufacturer recommends cutoff values of optical density of either greater than 35 or greater than 50. Preliminary data 15 indicate that the density of the non-treponemal line correlates with RPR titres. Laboratory technicians at Lihir Medical Centre did TPHA (Human Diagnostics, Wiesbaden, Germany) and the qualitative and quantitative RPR test (Human Diagnostics) and took measurements according to the manufacturers' instructions. The laboratory technicians who did the standard tests and read the results were unaware of the results of the DPP assay.

Statistical analysis
We calculated that a sample size of at least 579 people would give 80% power to estimate overall sensitivity and specifi city of the DPP assay in our study. We based our calculation on the expected sensitivity and specifi city of the DPP assay of 95%, 15 fi xed precision of the point estimate of 3%, and an estimated frequency of a positive TPHA or RPR test, or both, in the study population of 65%. 18 To calculate sensitivity and specifi city of the DPP T1 (treponemal line) assay, we compared DPP T1 assay positivity identifi ed by the naked eye with results of TPHA. To ascertain sensitivity and specifi city of the DPP T2 (nontreponemal line) assay, we compared DPP T2 assay positivity identifi ed by either the naked eye or automated reader (at recommended cutoff s of >35 or >50) with results of the RPR test. We did additional analyses to assess whether accuracy of results was consistent across subgroups defi ned according to disease stage (primary or secondary) or type of specimen (plasma or whole blood). We also compared the DPP T2 assay with RPR titres of 1:8 or higher, which have been judged the marker for true infection in past yaws studies. 20 We used Fisher's exact test to compare the sensitivity of the DPP T2 assay between RPR titres of 1:4 or less and 1:8 or greater.
We used a log-normal regression model to test whether intensity of the DPP T2 (non-treponemal line) point-ofcare assay measured by automatic reader grew with increasing RPR titre. To establish whether the automated reader could augment the performance of the DPP T2 assay, we used McNemar's test to compare the sensitivity and specifi city obtained by the reader device with values obtained by the naked eye. We assessed the overall performance of the DPP T2 assay versus the RPR test by receiver-operating characteristic analyses. We estimated the empirical optimum cutoff value for the DPP T2 assay with the Liu method, 21 which maximises the product of the sensitivity and specifi city, and with the nearest to (0,1) method, which fi nds the cutpoint on the receiveroperating characteristic curve closest to (0,1). We used the optimum cutoff value to estimate sensitivity and specifi city of the DPP T2 assay and compared values with those ascertained at a cutoff of greater than 35 or greater than 50, which were proposed by the manufacturer. To compare sensitivity and specifi city at diff erent cutoff s, we used McNemar's test.
We compared mean diff erences in DPP optical density at several timepoints using the negative binomial generalised estimating equation (GEE) populationaveraged regression model. We did statistical analyses with Stata version 13.0. All tests were two-tailed, and we judged p less than 0·05 signifi cant.
This study is registered with ClinicalTrials.gov, number NCT01841203.

Role of the funding source
Chembio Diagnostic Systems provided the DPP assay for use in the study, but otherwise, the funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The cor responding author had full access to all the data in the study and had fi nal responsibility for the decision to submit for publication.

Results
707 children were enrolled, of whom four withdrew from the study because of their unwillingness to undergo venepuncture. Of the 703 individuals tested, 368 (52%) were boys and 335 (48%) were girls; the median age of participants was 11 years (IQR 7-14). 504 participants were recruited from Lihir Island (group 1) and 199 were recruited from Karkar Island (group 2). 478 (68%) participants were suspected to have primary yaws and 225 (32%) had signs of secondary yaws. The recruitment and testing algorithm for participants is presented in fi gure 1.
504 plasma samples (group 1) and 199 whole-blood specimens (group 2) were collected and used for  Table 2 presents a comparison of both treponemal and non-treponemal DPP assays (T1 and T2) read by naked eye versus standard tests. The overall sensitivity of the DPP T1 assay was 88·4% and specifi city was 95·2%, compared with TPHA. The DPP T2 assay showed a sensitivity of 87·9% and a specifi city of 92·5% versus the RPR test. The ability of the DPP assay to detect both treponemal and non-treponemal (T1 and T2) antibodies, compared with the combination of positive TPHA and high-titre RPR test, showed an overall sensitivity of 93·9% (95% CI 89·5-96·8) and specifi city of 83·6% (80·1-86·7).
In prespecifi ed subgroup analyses (table 2), the sensitivity of the DPP T2 assay for children with primary yaws was signifi cantly higher than that for individuals with secondary yaws (diff erence of proportions 12·6%, 95% CI 4·0-22·8; p=0·005). The two types of specimenplasma and whole blood-had similar sensitivities and specifi cities for detection of treponemal and non-treponemal antibodies. Table 3 shows the sensitivity of the DPP assay to detect non-treponemal (T2) antibodies, according to RPR titre and by clinical stage and type of specimen. Among specimens with a high RPR titre (≥1:8), the overall sensitivity of the DPP T2 assay increased to 94·1% and sensitivity was signifi cantly higher in all subgroups, compared with specimens with lower RPR titres (≤1:4).
The area under the receiver-operating characteristic curve for the DPP T2 assay using automatic reading was 0·947. The optimum cutoff , identifi ed by both the    Liu method and the nearest to (0,1) method, was an optical density greater than 35, which gave a sensitivity of 89·8%, specifi city of 90·5%, and correct classifi cation of 90·0%. The sensitivity of T2 was increased slightly by use of automatic readers compared with the naked eye when the optical density cutoff was set at greater than 35, but it was lower when the cutoff was set at greater than 50 (table 4). 208 children in group 1 who had a positive DPP T1 assay and a reaction on the DPP T2 assay at an optical density greater than 35 were selected for follow-up, to monitor changes in the DPP T2 assay over time. 26 (13%) individuals could not be traced; the remaining 182 were included in subsequent comparative analyses at 3 and 6 months after treatment. The intensity of the DPP T2 assay decreased signifi cantly after treatment (p<0·0001; fi gure 3), with mean values at baseline of 390·4 (95% CI 344·1-436·7), at 3 months of 101·5 (80·3-122·7), and at 6 months of 31·5 (21·6-41·5). Rates of seroreversion after treatment (ie, optical density becoming negative, cutoff ≤35) were 47·8% (95% CI 40·5-55·1) at 3 months and 78·6% (72·6-84·6) at 6 months. If a fourfold reduction in optical density by 6 months were a second criterion for serological cure (after optical density ≤35), 173 (95%) of 182 participants would be judged cured at 6 months. Nine (5%) patients did not meet either criterion for serological cure: two (1%) had a sustained increase in optical density, suggesting treatment failure; and seven (4%) had an initial fourfold decline in optical density at 3 months followed by an increase in the titre, suggesting reinfection after cure. 38 samples were retested at the independent microbiology unit in Australia. Agreement was recorded for 37 (97%) TPHA and 35 (92%) RPR tests. Two (5%) discrepant samples for RPR were read as negative in the Lihir Medical Centre laboratory and were weakly positive (ie, titre 1:2) in the reference laboratory.

Discussion
In the communities we studied, a rapid and simple point-of-care DPP assay detected T pallidum pertenue infection with high specifi city and sensitivity. For all children included in our study, both the treponemal and non-treponemal DPP assays had high sensitivity and specifi city, and these values were similar in subgroups defi ned according to clinical characteristics and type of specimen tested (whole blood vs plasma).
The sensitivity of the DPP treponemal assay for yaws was slightly lower than expected (for syphilis it is more than 94-96%, vs 88% in our study), but the sensitivity of the non-treponemal assay was similar to that seen in syphilis (about 88%). 15,16 When we excluded children with low-titre RPR tests (ie, ≤1:4) from our analysis (102 [33%] of 305 RPR-positive), the sensitivity of the DPP T2 assay rose to 94%. These same specimens with low-titre RPR tests typically had low-intensity values of treponemal antibody, resulting in reduced sensitivity of the DPP T1 assay, but sensitivity rose to 97·9% in patients with an RPR titre of 1:8 or higher, suggesting that these individuals might have had latent yaws (infected a long time ago), with skin lesions caused by other infectious agents. 22 During a 6-month follow-up period, using an automated reader to measure the intensity of the non-treponemal rapid test line, we showed that DPP T2 optical density progressively fell after yaws treatment. Either seroreversion or a fourfold reduction in optical density was attained in 95% of patients 6 months after treatment. The optical density values of the nontreponemal assay have a similar response to the standard RPR titres that usually fall within 6-12 months.
Our study has several strengths. First, we assessed the DPP assays in two diff erent well-characterised communities of Papua New Guinea with known high prevalence of yaws. Second, we implemented standard external quality-control procedures for RPR testing and TPHA at an independent reference laboratory, and reproducibility was good. Finally, in addition to visual reading of results, we used an automatic reader to measure the intensity of DPP test lines and provide an objective quantitative or qualitative result by setting a cutoff criterion. The automatic reader was helpful to confi rm the reading obtained by the naked eye, and its use showed a small improvement in performance of the DPP assay. Measurement with the automatic reader has the   potential to monitor changes in density of non-treponemal antibodies (corresponding to titre) and allow for quantitative serological follow-up. However, electricity requirements and costs could be a barrier to routine use of this automated reader in resource-limited settings in yaws-endemic countries. The prevalence of active and latent yaws in this cohort and in other communities in the Pacifi c Islands is very high. 19,[23][24][25] About half the individuals screened at the community level who showed symptoms of yaws had dually positive TPHA and RPR serology. Moreover, 15% of people had fi ndings consistent with past or treated infection-ie, they were positive by treponemal assay but negative with non-treponemal testing. The high proportion of individuals with a discordance in treponemal and non-treponemal results highlights the limitations of using a treponemal test alone for yaws surveillance. The value of the new DPP test lies in the nontreponemal part, which can give immediate and accurate results for a diagnosis of active and untreated yaws infection rather than rely on a distant laboratory that is able to do RPR testing. Moreover, as reported in our study, sensitivity and specifi city of the DPP assay did not diff er between whole-blood samples and plasma specimens. The ability to use whole blood from a fi nger stick provides a point-of-care solution for yaws eradication. This advance is important for a disease that is typically prevalent in poor, isolated, rural settings in tropical areas.
For countries aiming to eradicate yaws, programmes should account for the variability of performance of the DPP assay, which is associated with low-titre RPR positivity. Furthermore, they should build in initiatives such as detection of new infectious cases and post-zero case surveillance.
A dual point-of-care test could be used before rounds of mass treatment; at this stage, the DPP test could identify dually positive patients with skin ulcers who have latent yaws and a lesion caused by bacteria other than T pallidum pertenue. However, after mass treatment rounds, the number of asymptomatic seroreactors will fall sharply and reliability of the DPP test to help in the diagnosis of new infectious cases will rise inversely. Our follow-up results 6 months after treatment, with a high proportion of seroreversion, support use of the DPP assay for active detection of new yaws cases after mass treatment. During resurveys, all people with ulcers should be screened with the DPP test and, if dually positive, both the index case and all their contacts should receive treatment with azithromycin. Those with ulcers but who are not positive on the DPP assay could be given syndromic treatment and followed up further.
For post-zero case surveillance, when the prevalence of yaws falls to very low levels, the positive predictive value of the non-treponemal line will decrease and, with a specifi city of 92%, a moderate proportion of positive results will be false positives. A more specifi c test might be needed to ensure that transmission has been interrupted.

Systematic review
We searched PubMed up to April 1, 2013, with the terms: "yaws", "syphilis", "point-of-care", "non-treponemal", and "rapid plasma reagin". We restricted our search to publications in English. We selected studies that assessed the accuracy of point-of-care tests in identifying the presence of non-treponemal antibodies in people with yaws. We found three studies [15][16][17] that showed accuracy of a new, dual, point-of-care immunoassay detecting non-treponemal and treponemal antibodies in syphilis. However, we did not identify any study that explored yaws diagnosis with a non-treponemal point-of-care assay.

Interpretation
Our results provide substantial evidence that the new, point-of-care, Dual Path Platform (DPP) Syphilis Screen and Confi rm assay can identify treponemal and non-treponemal antibodies in people with suspected clinical yaws. This fi nding represents a potentially useful advance in yaws diagnosis and control.
If our fi ndings are confi rmed in subsequent studies, and the DPP assay were to become widely available at an aff ordable price, it could help programmes to target yaws cases and contacts reliably and to identify new cases after mass treatment. Overall, the rate of overtreatment would be diminished considerably with use of the DPP non-treponemal and treponemal point-of-care assay, compared with the number of aff ected individuals who would be detected if only a treponemal point-of-care test were used. WHO should advocate for making this test available in yaws-endemic countries as part of the renewed eradication eff orts.

Contributors
TA, WH, QB, and OM designed the study. RW, RP, PM, and OM supervised fi eldwork and gathered data and samples. TA and L-NL were primarily responsible for serological studies. OM and EdL did statistical analyses. TA, DM, and OM wrote the fi rst draft of the report, with revisions and input from SVB, JE, WK, CM, QB, and PLA. All authors contributed to revisions and approved the fi nal version.

Declaration of interests
JE is employed by the manufacturer of the DPP assay, Chembio Diagnostic Systems. All other authors declare no competing interests.