Figures
Abstract
Background
HIV infection is a concern in the army troupes because of the risk behaviour of the military population. In order to allow regular access to CD4+ T cell enumeration of military personnel as well as their dependents and civilians living with HIV, the Senegalese Army AIDS program is implementing PIMATM Alere technology in urban and semi-urban military medical centres. Validation such device is therefore required prior their wide implementation. The purpose of this study was to compare CD4+ T cell count measurements between the PIMATM Alere to the BD FACSCountTM.
Methodology
We selected a total of 200 subjects including 50 patients with CD4+ T-cells below 200/mm3, 50 between 200 and 350/mm3, 50 between 351 and 500/mm3, and 50 above 500/mm3. CD4+ T-cell count was performed on venous blood using the BD FASCountTM as reference method and the PIMATM Point of Care technology. The mean biases and limits of agreement between the PIMATM Alere and BD FACSCountTM were assessed with the Bland-Altman analysis, the linear regression performed using the Passing-Bablok regression analysis, and the percent similarity calculated using the Scott method.
Results
Our data have shown a mean difference of 22.3 cells/mm3 [95%CI:9.1–35.5] between the BD FACSCountTM and PIMATM Alere CD4 measurements. However, the mean differences of the two methods was not significantly different to zero when CD4+ T-cell count was below 350/mm3 (P = 0.76). The Passing-Bablok regression in categorized CD4 counts has also showed concordance correlation coefficient of 0.89 for CD4+ T cell counts below 350/mm3 whilst it was 0.5 when CD4 was above 350/mm3.
Citation: Faye B, Mbow M, Cheikh Seck M, Mbengue B, Wade D, Camara M, et al. (2016) Evaluation of PIMATM CD4 System for Decentralization of Immunological Monitoring of HIV-Infected Patients in Senegal. PLoS ONE 11(5): e0154000. https://doi.org/10.1371/journal.pone.0154000
Editor: Alan Landay, Rush University, UNITED STATES
Received: January 17, 2016; Accepted: March 23, 2016; Published: May 11, 2016
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work. However, the HIV/AIDS Prevention Program (DHAPP) of the US Department of Defense (URL: https://www.aids.gov/federal-resources/federal-agencies/department-of-defense/) provided laboratory equipments that have been used to perform this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
HIV infection is a concern for the Senegalese army forces because of the high mobility of soldiers frequently engaged in external peace keeping missions, the task of risk of the military population, and the high prevalence of HIV in Casamance post conflict area [1] where there is a high concentration of the military population. The fight against AIDS in the Army as well as in the general population is in line with the safety of the troops. In an effort to support people living with HIV (PLHIV), efforts have been made to better access the clinical outcomes and ensure accurate diagnostics.
CD4+ T-cell enumeration is frequently used to assess the immunity level of HIV-positive patients, determining their eligibility for antiretroviral therapy (ART) [2, 3]. In most clinical settings, CD4+ T-cell enumeration is performed using flow cytometry-based systems, such as FACSCountTM (Becton Dickinson, San Jose, CA), and is considered the reference method because of its precision, accuracy, and reproducibility [4–8]. However, costly maintenance, the need for well-trained laboratory staff and a cold chain to ship and store reagents can limit its use. Based on these field challenges, the AIDS program of the Senegalese Army Forces, in collaboration with Senegalese National AIDS Program and United States Department of Defense HIV/AIDS Prevention Program (DHAPP), is implementing the PIMATM Point-of-care (POC) CD4+ T-cell enumeration technology (Alere, Jena, Germany) in urban and semi-urban military bases and medical facilities where military personnel, their dependents and civilians living with HIV are being cared for and treated at these sites. Adoption of such non-flow cytometry-based CD4+ T-cell counting device will allow regular access to CD4+ T-cell enumeration in rural and semi-urban sites in Senegal. Utilizing the PIMATM Point-of-care (POC) CD4+ T-cell enumeration technology will also supports the National and Army AIDS programs strategies to control HIV infection and its transmission. However, since laboratory and field evaluation of the CD4 systems is part of the prequalification of a diagnostic program, validation of POC technologies is therefore required prior their wide implementation [9].
The purpose in this study was to compare T-cell count measurements between the PIMATM CD4 (Alere, Jena, Germany) to the BD FACSCountTM which has been extensively validated in resource-limited settings [10–11].
Material and Methods
Study population and sites
Study participants consisted of adults HIV-infected patients who were being followed up through the Senegalese National AIDS Program and Army AIDS Program from July 2014 to May 2015. Whole blood samples were collected in K3 EDTA-containing tubes from 200 individuals including 183 HIV-infected patients and 17 HIV-negative volunteers who agreed to participate in the study. These facilities include the military bases of Tambacounda, Ziguinchor, Kolda as well as the Military Hospital of Ouakam (MHO) which is the main military health facility in Senegal located west of the Dakar. Subjects were selected based on their CD4+ T-cell count as follows: 50 patients below 200/mm3 50 patients between 200 and 350/ mm3, 50 patients between 350 and 500/ mm3, and 50 patients above 500/ mm3.
CD4+ T-cell enumerations were performed at the medical biology laboratory of the MHO that hosts the laboratory platform of the Senegalese Military AIDS Program.
Ethics statement
This study was approved by the National Ethical Committee of the Ministry of Health of Senegal. The ethics committee waived the need for written informed consent from the study participants since we used excess of blood samples from anonymous participants who underwent their routine CD4+ T-cell count.
CD4+ T-cell enumeration
CD4+ T-cell counts were assessed in whole blood samples within 4 hours of blood collection using the FASCountTM (Becton Dikinson) analyzer and the PIMATM Point of Care technology.
The BD FACSCountTM, equipped with a single platform possesses a green laser combined with built-in software uses two-color monoclonal antibodies (mAb) reagents in twin tubes containing calibrated beads, with additional control beads. Briefly, 50μl of EDTA whole blood was added to the BD FACSCountTM reagent tubes containing anti-CD3-PE, and anti-CD4-PE-Cy5 or anti-CD8-PE-Cy5. The tubes were capped, vortexed and incubated in the dark at room temperature for 1 hour. After incubation, 50ul of fixative solution was added to reagent tubes, and samples were run on the BD FACSCountTM instrument.
The BD FACSCountTM machine is certified by the Quality Assessment and Standardization of Immunological Measures Relevant to HIV/AIDS (QASI) with the BD FACSCountTM reagents that only provide absolute CD4 counts.
Similarly, the CD4+ T-cell measurement with the PIMATM POC was performed according to manufacturer’s procedure. The samples were run only after the normal and low value control cartridges gave acceptable values. Twenty five microliters of venous blood sample collected in EDTA tube were introduced into the PIMA disposable cartridge containing anti-human CD3-dye1 and CD4-dye2 monoclonal antibodies. The collector was then removed and the cartridge immediately inserted into PIMA analyzer.
Statistical analysis
Data were entered in Microsoft Access and then exported to SPSS version 17 (IBM), MedCalc 10.0.2.0 (MedCalc Software, Mariakerke), and GraphPad Prism version 5.00 for Windows (GraphPad Software, San Diego California USA) for statistical analysis and graphing. The differences between the two measurements as well as the mean differences of these measurements were computed in SPSS software. One-sample T-test was used to assess the difference between the two measurements by comparing the mean differences to zero. The mean biases and the limits of agreement (LOA) were assessed with the Bland-Altman analysis using the mean differences between the two measurements and the 95% confident limits of the measurement (LOA = mean ± 1.96 SD). The linear regression was performed using the Passing-Bablok regression analysis. Percent similarity between the PIMATM Alere and BD FACSCountTM was calculated for each sample following the formula: average of methods A and B x 100)/method A (where method A = reference method (BD FACSCountTM) and method B = method to evaluate (PIMATM Alere)) [12].
Results
Study population
A total of 200 adults were elected for the present study. The median age was 36 years (range: 19–69) and the female sex represented 69% of the study population. The median (min-max) absolute CD4 counts provided by BD FACSCount and PIMATM alere were 349 cells/mm3 (1–2000) and 332 cells/mm3 (9–2221) respectively.
Difference between the BD FACSCountTM and PIMATM Alere CD4 measurements
The median CD4 counts obtained using PIMATM Alere analyzer were compared with those obtained from the BD FACSCountTM for the following count categories <200, 200–350, 351–500 and >500 cells/mm3. For all CD4 ranges, the differences in median CD4 counts were not significant between the two methods (all p-values > 0.05) (Fig 1).
Categories of absolute CD4+ T-cells counts (< 200/mm3, [200-350/mm3], [351-500/mm3], and > 500/mm3, in before-after scatter plots comparing the PIMATM Alere and the BD FACSCountTM are shown. Data are shown as median values. P-values were calculated in SPSS 20 using nonparametric Mann-Whitney U test and the graphing was performed using GraphPad Prism software version 5.00.
We also investigated the difference between the two measurements by comparing the mean differences between the two methods to zero using the one-sample T test. Our data have shown that the mean difference of 22.3 cells/mm3 [95%CI:9.1–35.5] that was significantly different from zero, indicating that overall, the measurements by the two methods were not similar (P = 0.001) (Table 1). Using the categorized CD4 cell counts, the mean differences were -4,0 [95%CI:-11.9–3.9], -2,7 [95%CI:-20.9–15.4], 19.9 [95%CI:1,6–38,3], and 76,1 [95%CI:32,9–119.4] for CD4 counts < 200/mm3, between 200 and 350/mm3, between 351 and 500/mm3, above 500/mm3, respectively. The mean differences of the two measurements to zero was not significantly different for CD4 cell count levels below 200/mm3 and between 200 (P = 0.31) and 350/mm3 (P = 0.76), indication that there was an agreement for the cell count levels between the two measurements for CD4 counts below 350/mm3. Conversely, there was no agreement in the cell count when CD4 counts were between 350 and 500/mm3 (P = 0,034) or above 500/m3 (P = 0.001) (Table 1).
The difference and mean differences of the counts obtained were plotted using the Bland-Altman method to visualize the bias, i.e. the random fluctuation of measurements around the mean difference line. We found that the average discrepancy between the methods (bias) was higher when the CD4 count was above 350/mm3 (Fig 2). In other words, the difference between the two methods became larger as the average increased.
The x-axis represents the average of CD4 count between the PIMA and the BD FACSCountTM, and the y-axis represents the bias (difference) between the PIMATM Alere and the BD FACSCountTM. The solid blue line represents the mean of the difference between the two measurements, and the light black lines represent the upper and lower limits of agreement (ULA: mean differences plus and 1,96 x standard deviation of the mean difference; LLA: mean differences minus and 1,96 x standard deviation of the mean difference). Legend for CD4 categories: 1: CD4 T-cells < 200/mm3; 2: CD4 T-cells between 200 and 350/mm3; 3: CD4 T-cells between 351 and 500/mm3; 4: CD4 T-cells above 500/mm3.
The Bland Altman plot analysis showed an absolute bias of -22 (with lower and higher agreement limits of -208 and 163 respectively) indicating that overall, the PIMA underestimated the BD FACSCountTM by 22 cells/mm3. Calculation of the relative bias has shown an inter-essay variation of -0.2%. The Bland-Altman plots also showed that the higher the CD4 T-cell counts were, the wider the limits agreements became (Fig 3; Table 2).
Bland-Altman plots of (A) CD4 T-cells < 200/mm3, (B) between 200 and 350/mm3, (C) between 350 and 500/mm3, and (D) above 500/mm3 according the CD4 cell count levels. For each plot, the x-axis represents the average of CD4 count between the PIMA and the BD FACSCountTM, and the y-axis represents the bias (difference) between the PIMATM Alere and the BD FACSCountTM. The solid red line represents the mean of the difference between the two measurements, and the light black lines represent the upper and lower limits of agreement (ULA: mean differences plus and 1,96 x standard deviation of the mean difference; LLA: mean differences minus and 1,96 x standard deviation of the mean difference)
No proportional bias was found when the CD4 count was below 200/mm3 (P = 0.29). However, there were biases between the two measurements when CD4 counts were above 200mm3 (p-value significant for all CD4 levels above 200/mm3) (Table 1 and Table 2).
The linear regression analysis did not show a proportional bias between the two measurements (Fig 4). The Passing-Bablok regression analysis between the two measurements showed an intercept of 11 cells/mm3 and a slope of 0.93 (Table 2). The corresponding concordance correlation coefficient was 0.94 for PIMATM Alere, showing a good correlation and concordance between the two methods (Table 2).
The x-axis represents CD4 counts provided by the BD FACSCountTM reference and the y-axis represents the CD4 counts provided by the PIMATM CD4. The solid line represents the regression line, and the dashed line represents the line y = x. The linear equation of the regression is y = 0.9347 x + 11.
The Passing-Bablok regression analysis using categorized CD4 counts also showed concordance correlation coefficient of 0.89 when CD4+ T-cell counts were below 350/mm3 while it was 0.5 when CD4 was above 350/mm3.
Discussion
As a result of intensive HIV testing campaigns in order for people to know their status, and the well-established importance of getting people living with HIV (PLHIV) in care and offered treatment, Senegal has one of the lowest HIV prevalence in Sub-Saharan Africa. In an effort to improve and decentralize care for PLHIV, there is a needed of better access to CD4+ T-cell enumeration. In order to allow regular access to CD4+ T-cell enumeration in rural and semi-urban sites in Senegal as well as respond to the National AIDS program and WHO strategy to control HIV infection, the Senegalese Army AIDS program is implementing PMATM Alere CD4+ T-cell enumeration technology in urban and semi-urban military medical centre.
The BD FACSCountTM has been the first instrument offering affordable and cheaper T-cell counts for biological monitoring of HIV-positive patients. However, its high supply and maintenance costs inspired the Senegalese AIDS National Program and the Army AIDS Program to progressively switch toward affordable new POC CD4 counting instrument including the PIMATM Alere which is able to respond to this demand. Due to the limited precision and accuracy that all instruments might have and that might lead to discordant results, we aimed in this study to validate the PIMATM instrument using the FACSCountTM as reference instrument. We found that for low CD4 values, the PIMATM counter provided concordant results compared to the BD FACSCountTM.
Our data showed that the PIMATM analyzer gave comparable median CD4 counts with the reference methodology in different ranges, including below 200, 200–350, 351–500 and above 500 cells/mm3. There was no significant difference in the median CD4 counts between the PIMATM and the FACSCountTM. This is consistent with the findings reported by Thakar et al. who did not find any significant difference in median CD4 counts obtained by the reference method and the PIMATM analyzer [13].
However using more conventional comparison method of two quantitative measurements, we assessed the mean difference between the two measurements that appeared significantly different from zero using One-sample T-test. This indicates that there was no perfect agreement between the PMATM and FACSCountTM measurements. After categorizing CD4+ T-cell counts, we found that such difference was not significant for CD4 ranges below 350/mm3, highlighting a good agreement between the two measurements for low CD4 counts.
The analysis of the agreement between FACSCountTM and PIMATM Alere CD4 measurement have shown an absolute bias of -22 cells, indicating that overall, the PIMATM slightly underestimates the gold standard by 22 cells/mm3. This is consist with Myer et al. findings in HIV-positive pregnant women showing the PIMATM POC test to slightly underestimate CD4 cell count relative to flow cytometry based systems by 23 cells/mm3, with limits of agreement from -129.2 to 174.6 cells/mm3 [14]. As we reported, the majority of studies comparing the Alere PIMATM analyzer using venous specimens have found that the POC methods underestimate laboratory-based flow cytometry methods [15,16] despite providing still a good correlation with conventional technologies [17]. Nevertheless, the relative bias between the PIMATM and reference methodology was very low, suggesting a good agreement between the two measurements. Similar low relative bias has also been reported in studies comparing CD4+ T-cell counts between the PIMATM Alere and the BD FACSCountTM [13].
Through the Passing-Bablok regression analysis, we found a good concordance correlation coefficient between the methods for low and medium CD4 values. This is in line with previous findings showing that the PIMATM CD4 was found to be less precise than the flow cytometry based systems, in particular for CD4 counts below 200 cells/mm3 [18]. Previous studies have reported that the PIMATM device is comparable and interchangeable with the existing CD4 enumeration platforms [13, 15, 19, 20]. In our study, despite the PIMA Counter showing good concordance and correlation with BD FACSCountTM, we did only find such evidence in low and medium CD4 counts. This discrepancy might however reflect the significant contribution of experiment conditions.
Consistent with our results, when CD4 counts were above 500/mm3, the PIMATM Alere tended to misclassify patients by 80 cells/mm3 (with agreement limits from -376 to 216 cells/mm3) toward an underestimation compared with the BD FACSCountTM. This suggests that using the PIMA technology, patients who need treatment may not qualify for ART under current guidelines. Nevertheless, the bias was low (< 20 cells/mm3) for CD4 counts below 350/mm3.
It has also been reported that the sensitivity and specificity of the PIMATM Alere was 83.1% and 92.2% respectively when BD FACSCaliburTM was used as gold standard, and 79.4% and 83.4% respectively when BD FACSCountTM was considered as reference method [21]. Moreover, the same study has shown that the PIMATM and BD FACSCountTM coefficients of repeatability were very different, at 179.2 cells/ml and 67.4 cells/ml respectively, indicating that the tests on the BD FACSCountTM were more repeatable. These results and ours do not fully support that the PIMATM device is interchangeable with the FACSCountTM CD4 enumeration platform. Since the purpose of point-of-care technologies is to provide wider access to a CD4+ T-cell enumeration service in remote rural settings, they may not provide the same sensitivity, specificity and accuracy as the ‘gold standard’ reference tests. Moreover, the principles in the tests themselves, i.e. fluorescence optic imaging versus flow cytometry based system might reflect differences in the precision regarding certain ranges of CD4 counts. Overall, if operators’ contribution on such discrepancies is dismissed, it therefore appears that the PIMATM technology might need further refinement to improve repeatability. Because of its characteristics and particular interest for use in small health from resource-limited settings, improvement of the PIMATM Alere is nevertheless a need, especially in light of the new World Health Organizations recommendations on starting treatment as soon as possible [22].
Overall, we have demonstrated that for low CD4+ T-cell counts, the results from the PIMATM Alere obtained from venous blood can provide accurate data which a good agreement with the FACSCountTM. Despite PIMATM Alere CD4 system can be operated easily in resource-limited settings, its wider implementation will require deeper evolution and/or further refinement of its system so that to reach perfect concordance between the PIMA and the existing reference systems.
Acknowledgments
The authors would like to thank the laboratory staff of the Military Hospital of Ouakam as well as patients and volunteers for their participation in the study. We are grateful to the HIV/AIDS Prevention Program (DHAPP) of the US Department of Defense that provided laboratory equipments used to perform this study. We also express our deep gratitude to, Dr Abdoulaye Sidibe Wade from the National AIDS program, William Conquest, and Absa Diop from the Department of Defense of the US embassy in Dakar who contribute to equipment and reagent purchasing.
Author Contributions
Conceived and designed the experiments: A. Dièye BF MM. Performed the experiments: SGD MM. Analyzed the data: MM BF DW MC. Contributed reagents/materials/analysis tools: MPG A. Djibo BF. Wrote the paper: MM BF A. Djibo MPG. Conceived and designed the study: BF A. Dièye MM. Reviewed critically the manuscript for important intellectual content: MCS BM DW MC CC BN A. Djibo MDSN TN MPG A. Dièye. Approved the final version to be published: A. Dièye MPG.
References
- 1.
Accessible HIV/AIDS prevention and care service for all: Handicap International UK. Retrieved from (2015). Available: http://www.handicap-international.org.uk/where_we_work/africa/senegal
- 2. Kagaayi J, Makumbi F, Nakigozi G, Wawer MJ, Gray RH, et al. WHO HIV clinical staging or CD4 cell counts for antiretroviral therapy eligibility assessment? An evaluation in rural Rakai district, Uganda. AIDS 2007; 21:1208–10. pmid:17502733
- 3. Jaffar S, Birungi J, Grosskurth H, Amuron B, Namara G, et al. Use of WHO clinical stage for assessing patient eligibility to antiretroviral therapy in a routine health service setting in Jinja, Uganda. AIDS Res Ther 2008; 5:4. pmid:18307778
- 4.
World Health Organization. HIV in the WHO African Region: Progress towards achieving universal access to priority health sector interventions, 2011 update.
- 5.
World Health Organization. Consolidated Guidelines on the use of Antiretroviral Drugs for Treating and Preventing HIV infection: Recommendations for a Public Health Approach. 2013.
- 6. Mandy FF, Nicholson JK, McDougal JS. Guidelines for performing single platform absolute CD4+T-cell determinations with CD45 gating for persons infected with human immunodeficiency virus. Centers for Disease Control and Prevention. MMWR Recomm Rep 2003; 52:1–13.
- 7.
World Health Organization. Antiretroviral therapy for HIV infection in adults and adolescents. Recommendations for a public health approach: 2010 revision.
- 8.
World Health Organization. Technical brief on CD4 technologies. 2010.
- 9. Hyle EP, Jani IV, Lehe J, Su AE, Wood R, Quevedo J, et al. The Clinical and Economic Impact of Point-of-Care CD4 Testing in Mozambique and Other Resource-Limited Settings: A Cost-Effectiveness Analysis. PLos Medicine 2014;
- 10. Strauss K, Hannet I, Engels S, Shiba A, Ward DM, et al. Performance evaluation of the FACSCount System: a dedicated system for clinical cellular analysis. Cytometry 1996; 26:52–59. pmid:8809481
- 11. Lopez A, Caragol I, Candeias J, Villamor N, Echaniz P, et al. Enumeration of CD4(+) T-cells in the peripheral blood of HIV-infected patients: an interlaboratory study of the FACSCount system. Cytometry 1999; 38:231–237 pmid:10516609
- 12. Scott LE, Galpin JS, Glencross DK. Multiple method comparison: statistical model using percentage similarity. Cytometry Part B (Clinical Cytometry) 2003; 54B:46–53.
- 13. Thakar M, Mahajan B, Shaikh N, Bagwan S, Sane S, Kabra S, et al. Utility of the point of care CD4 analyzer, PIMA, to enumerate CD4 counts in the field settings in India. AIDS Research and Therapy 2012; 9:26. pmid:22998738
- 14. Myer L, Daskilewicz K, McIntyre J, Bekker LG. Comparison of point-of-care versus laboratory-based CD4 cell enumeration in HIV-positive pregnant women. Journal of the International AIDS Society 2013, 16:18649. pmid:24044627
- 15. Mtapuri-Zinyowera S, Chideme M, Mangwanya D, Mugurungi O, Gudukeya S, et al. Evaluation of the PIMA point-of-care CD4 analyzer in VCT clinics in Zimbabwe. J Acquir Immune Defic Syndr 2010; 55:1–7 pmid:20622679
- 16. Sukapirom K, Onlamoon N, Thepthai C, Polsrila K, Tassaneetrithep B, Pattanapanyasat K. Performance evaluation of the Alere PIMA CD4 test for monitoring HIV-infected individuals in resource-constrained settings. J Acquir Immune Defic Syndr 2011; 58(2):1417)
- 17. Jani IV, Peter TF. How point-of-care testing could drive innovation in global health. N Engl J Med 2013; 368: 2319–2324). pmid:23758238
- 18. Jani IV, Sitoe NE, Chongo PL, Alfai ER, Quevedo JI, et al. Accurate CD4 T-cell enumeration and antiretroviral drug toxicity monitoring in primary healthcare clinics using point-of-care testing. AIDS 2011; 25:807–812. pmid:21378535
- 19. Diaw PA, Daneau G, Coly AA, Ndiaye BP, Wade D, et al. Multisite evaluation of a point-of-care instrument for CD4(+) T-cell enumeration using venous and finger-prick blood: the PIMA CD4. J Acquir Immune Defic Syndr 2011; 58:e103–111. pmid:21909029
- 20. Herbert S, Edwards S, Carrick G, Copas A, Sandford C, et al. Evaluation of PIMA point-of-care CD4 testing in a large UK HIV service. Sex Transm Infect 2012; 88:413–417. pmid:22544309
- 21. Mwau M, Adungo F, Kadima S, Njagi E, Kirwaye C, Abubakr NS, et al. Evaluation of PIMATM Point of Care Technology for CD4 T-cell Enumeration in Kenya. Plos One 2013; 6:e67612).
- 22.
World Health Organization. 2015 ARV guidelines. Available: http://www.who.int/hiv/mediacentre/news/arv2015_csocall/en/