To err is human, to correct is public health: a systematic review examining poor quality testing and misdiagnosis of HIV status

Abstract Introduction: In accordance with global testing and treatment targets, many countries are seeking ways to reach the “90-90-90” goals, starting with diagnosing 90% of all people with HIV. Quality HIV testing services are needed to enable people with HIV to be diagnosed and linked to treatment as early as possible. It is essential that opportunities to reach people with undiagnosed HIV are not missed, diagnoses are correct and HIV-negative individuals are not inadvertently initiated on life-long treatment. We conducted this systematic review to assess the magnitude of misdiagnosis and to describe poor HIV testing practices using rapid diagnostic tests. Methods: We systematically searched peer-reviewed articles, abstracts and grey literature published from 1 January 1990 to 19 April 2017. Studies were included if they used at least two rapid diagnostic tests and reported on HIV misdiagnosis, factors related to potential misdiagnosis or described quality issues and errors related to HIV testing. Results: Sixty-four studies were included in this review. A small proportion of false positive (median 3.1%, interquartile range (IQR): 0.4-5.2%) and false negative (median: 0.4%, IQR: 0-3.9%) diagnoses were identified. Suboptimal testing strategies were the most common factor in studies reporting misdiagnoses, particularly false positive diagnoses due to using a “tiebreaker” test to resolve discrepant test results. A substantial proportion of false negative diagnoses were related to retesting among people on antiretroviral therapy. Conclusions: HIV testing errors and poor practices, particularly those resulting in false positive or false negative diagnoses, do occur but are preventable. Efforts to accelerate HIV diagnosis and linkage to treatment should be complemented by efforts to improve the quality of HIV testing services and strengthen the quality management systems, particularly the use of validated testing algorithms and strategies, retesting people diagnosed with HIV before initiating treatment and providing clear messages to people with HIV on treatment on the risk of a “false negative” test result.


Introduction
In the last decade, HIV testing services have been scaled-up substantially. In 2005, it was estimated that only 12% of people who wanted an HIV test were able to access testing; and that only 10% of people with HIV in Africa knew their status [1]. In contrast, between 2010 and 2014, more than 600 million people in 122 low-and middle-income countries received HIV testing [2], and as of 2015, approximately 60% of people with HIV were aware of their status [3]. Such scale-up has been possible through the expansion of providerinitiated testing and counselling and community-based testing programmes, which have routinized HIV testing and extended services to many people.
Rapid diagnostic tests (RDTs) have been instrumental to the scale-up of HIV testing, particularly in resource-limited settings where access to laboratory services is poor. RDTs have been shown to be highly accurate and can often provide a same-day diagnosis when used within a validated testing strategy (i.e. the order in which the tests are performed) and algorithm (i.e. the exact tests used within the testing strategy) according to high (≥5%) and low HIV prevalence (<5%), as recommended by the World Health Organization (WHO) [4][5][6]. Recent reports, however, have shown that HIV testing is not always conducted appropriately [7,8], and in some countries, quality systems have not kept pace with testing scale-up. According to a review of national HIV testing policies, less than 20% of testing strategies were consistent with WHO guidance, and only two included recommendations on retesting prior to the initiation of antiretroviral therapy (ART) [9]. In some cases, poor-quality testing has resulted in incorrect test results and the misdiagnosis of HIV status [10][11][12][13][14].
HIV misdiagnosis refers to any testing event where a diagnosis is missed, inappropriately delayed or incorrect (either false positive or false negative) [15]. Poor-quality HIV testing and misdiagnosis have negative consequences for individuals, families, communities, health workers and health services. False negative diagnoses represent missed opportunities to identify an HIV infection and link people to early treatment. False positive diagnoses may cause social and emotional harm and create mistrust of health workers and the test results they deliver. Without addressing HIV testing quality, new guidance offering same-day treatment to all people diagnosed with HIV [16] could lead to inappropriate ART initiation [11]. Once individuals are on treatment, because ART reduces antibody production and can cause seroreversion, for example, false negative test results, determining a person's true HIV status can be especially challenging [17,18].
We conducted this systematic review to assess the magnitude of misdiagnosis and to identify and describe poor HIV testing practices using RDTs, including those which may have led to incorrect test results and misdiagnosis.

Methods
We systematically searched for peer-reviewed articles published from 1 January 1990 to 19 April 2017 using a predefined search strategy in the following electronic databases: PubMed, CINAHL and EMBASE. All conferences of the International AIDS Society were searched from July 2001 through July 2016; the most recent Conference on Retroviruses and Opportunistic Infections (2014-2017) database were searched because past conference abstracts were unavailable. Conferences of the African Society of Laboratory Medicine (ASLM) were searched 2012-2016, as well as the ASLM website and other key global health websites (see supplementary information). We searched reference lists to identify additional literature. This process was repeated until no new citations were identified. Experts were also contacted to identify additional reports. No geographic restrictions were placed on the search, but the review was limited to studies published in English.
Studies were eligible if they used at least two RDTs and reported on HIV misdiagnosis, factors related to potential misdiagnosis or described quality issues and related to HIV testing error. Initial titles were screened by one investigator (VF) to determine eligibility. A second and a third screening was then carried out (VF, ST and CJ). All differences were resolved through consensus. Data from all sources were extracted and placed into standardized forms and verified in duplicate (VF and ST). CJ and NF assessed study quality (see supplementary data).
Potential factors relating to misdiagnosis were extracted from studies using defined categories: (a) clerical error (error in documenting and reporting information essential to a correct status); (b) user error (operator error collecting specimen, performing an HIV RDT or interpreting the result); (c) suboptimal testing strategy (errors related to the order in which specific RDTs are used, also known as a testing strategy); (d) poor management and supervision (lack of active quality management systems); (e) weak reactive results (faint lines appearing on test strips); and (f) additional factors including cross-reactivity, acute/early infection and testing among people on ART.
Other summary measures included: misdiagnosis rates (total number of false positive diagnoses reported over the total number of HIV-positive tests retested and reported using a specific testing algorithm and the total number of false negative diagnoses reported over the number of HIV-negative tests retested and reported using a specific testing algorithm). For studies exclusively among people diagnosed with HIV, reporting on false positive statuses, the total study population was used as the denominator.
For each study, rates of diagnostic error and misdiagnosis and corresponding 95% confidence intervals (CIs) were calculated, using Wilson's approach, and this was displayed graphically using forest plots [19][20][21]. All statistical analyses were conducted in STATA v13.0.

Results
Sixty-four studies reporting on misdiagnosis of HIV and factors potentially related to misdiagnosis were included in this review ( Figure 1 and Table 1).
Factors related to the quality of HIV testing and potential misdiagnosis Several factors, including HIV testing errors, were reported frequently (n = 131 times) across all included studies (see Table 2).
Fourteen studies reported challenges related to weak reactive test results, particularly difficulty with interpretation [8,10,24,27,30,32,36,38,40,42,44,48,62,74]. A study, which assessed the proficiency of laboratory technicians, found that specimens with very weak levels of HIV-1/2 antibodies were less accurately reported [40]. In Uganda, two studies found that the majority of false reactive results came from weak reactive RDTs [32,36]. A study from the UK that assessed the visual depiction of false reactive and true positive readings reported that most false reactive specimens had a fainter test line than true positive specimens   (Bile et al. 2017) did not report a specific factor or error related to misdiagnosis. Some studies reported multiple factors related to poor quality testing and factors that could be related to potential misdiagnosis. [48]. Two studies reported incorrect reading of weak reactive bands contributed to the misdiagnosis [10,11].
Eighteen studies reported on several other testing errors and factors potentially related to misdiagnosis. Nine of these studies reported cross-reactivity either between RDTs within an algorithm or with population and individual characteristics [10,22,24,25,27,32,38,56,60]. One study suggested that cross-reactivity between assays used within an algorithm resulted in false positive statuses [27]. Another hypothesized that cross-reactivity may present as weak reactive lines and thereby cause misdiagnosis [32]. Six studies [10,11,25,38,56,60] reported potential issues with RDTs interacting with characteristics of individuals undergoing testing [10,11,38], including having low levels of HIV-1/2 antibodies due to late stage HIV infection [56,60] and exposure of assays to adverse environmental conditions during storage and use [25,38].
Additionally, six studies reported that a proportion of false negative diagnoses were among people with a known HIV status who were on ART [14,16,39,44,47,58]; one of these studies was among children on ART retested using an oral fluid-based HIV RDT [58]. And three studies reported false negative results were due to patients testing in the window period [45] or with acute or early infection [16,44]. For instance, in South Africa, 0.04% (95% CI: 0.0-0.001) and 0.3% (95% CI: 0.1-0.4) of clients with a false negative diagnosis using serology tests were later found to have acute or early HIV infection after retesting with nucleic acid testing technologies [44].
Nearly all studies reporting false negative diagnoses also reported using a suboptimal testing strategy. Four studies in South Africa reporting false negative diagnoses reported that HIV testing was conducted with an algorithm using a first-line test with high specificity and poorer sensitivity [14,33,39,55]. According to one of these studies [14], between 2014 and 2016, the testing algorithm changed four times in an effort to address the high proportion of false negative diagnoses resulting from these algorithms.
Clerical and user errors [57], early/acute infection [16,44,45], presentation late in disease stage [56] and individuals with known HIV status on ART who sought retesting, or were retested using oral fluid-based RDTs [58], were also reported as factors contributing to false negative diagnostic errors [14,16,39,44,47]. In Zimbabwe, all the reported false negative diagnoses were among children on ART who were retested with an oral fluid-based HIV RDT [58]. In South Africa and Zambia, individuals on ART comprised 44% (26/ 59) and 14% (5/38) of false negative diagnoses, respectively [14]. In Mozambique, 88% (21/24) of all true HIV-positive clients with a false negative test result were confirmed to know their HIV status and 62% (13/21) were reportedly on ART [47]; reasons for retesting in study reportedly included users misunderstanding the question or hoping to receive health services and emotional or mental health issues.

Discussion
This review identified and described a number of diagnostic errors and poor HIV testing practices that may lead to misdiagnosis. Data on the magnitude of misdiagnosis was identified but limited, and no study could determine or quantify the exact cause(s) of misdiagnosis. Although no studies could determine and quantify the exact cause(s) of misdiagnosis, several identified the following factors to have strongly contributed: (1) suboptimal testing strategies, (2) poor management of supplies, (3) user errors including difficulty interpreting weak reactive lines and (4) retesting among people with known HIV status on ART.
No assay is perfect. False reactive and false non-reactive results are inevitable when using a single RDT and should be anticipated. However, the risk of misdiagnosis should be very low when a validated testing algorithm for high (≥5%) or low (<5%) prevalence settings is used [76]. In this review, we identified that many studies reporting diagnostic errors -both false positive and false negative -utilized suboptimal testing strategies which were not aligned to international guidance. Studies reviewed clearly showed the use of a tiebreaker strategy to rule-in HIV infection increases the likelihood of false positive statuses and possible misdiagnosis. This is concerning because a third of national testing strategies reviewed in 2015 recommended using a tiebreaker testing strategy [9].
In addition to adopting a proven testing strategy, national or regional validation is critical to determine which RDTs, and in which order, perform the best as a complete algorithm. As previously reported [38,[77][78][79][80][81][82][83], tests and algorithm performance vary across settings, often due to cross-reactivity caused by HIV subtypes, co-infections, comorbidities and possible environmental or population characteristics. Without validating a testing algorithm at a country or regional level, it would not be possible to fully understand the causes of poor performance. Furthermore, to ensure correct diagnoses, it is important to retest people diagnosed HIV positive before they enrol in care and ART. This is a cost-effective approach [84] which is increasingly critical as more people with HIV are being offered immediate treatment.
To ensure correct results, all staff providing HIV testing must be trained, certified and provided ongoing support and supervision. In several studies, this was not the case, and untrained and uncertified providers were performing HIV testing [7,72]. Training, including pre-service, in-service and periodic refresher training, is important to maintain and improve the quality of services. Participation in EQA schemes is another way to monitor performance and improve testing services. Several studies also reported user and clerical errors resulted from inadequate support, demanding workloads, burnout and high levels of stress [11,62,64,66]. Adequate support and supervision are critical to reduce stock outs which may contribute to the use of damaged or expired test kits, incorrect test kits and buffer. Sites should routinely assess and manage their supplies and human resource planning to prevent or reduce these circumstances.    reported proportion misdiagnosed by did not report full sample size. In studies where all participants were known to be HIV positive and/or on ART at the beginning of the study, the full study population was used as the denominator.  User error interpreting weak reactive lines was a common challenge which contributed to false positive results. To address this, specialized training for health workers and site-level standard operating procedures including the use of a "second-reader" to validate the correct interpretation of test results may be needed, as well as work with manufacturers to improve RDTs and instructions on how to interpret faint lines and weak control lines. Several studies hypothesized that weak reactive lines may be caused by other user errors, for example, misapplication of buffer and reading test results too early and cross-reactivity. Further investigation into the cause of weak reactive and other faint lines, and how they can be prevented, is needed.
False negative test results among people with HIV and on ART were observed and contributed to a substantial proportion of misdiagnoses [14,16,39,44,47]. While it is unclear why people on ART would seek retesting, some reports suggest it may be due to wanting to "check" or "confirm" one's HIV status and religious beliefs about being "cured" [85], as well as misunderstandings and emotional or mental health issues [47]. It is important for programmes and users to be aware of the potential risk of false negative results, as the presence of ART can lead to confusing test results and could result in individuals unnecessarily stopping treatment which could have dire individual and public health implications. As "treat all" policies are rolled out, it will be increasingly critical for programmes to address this issue and ensure clients and health workers are aware that testing individuals on ART is not recommended [76].

Strengths and limitations
This analysis is the first to bring together a diverse set of studies with the aim of identifying and describing suboptimal HIV testing practices and misdiagnosis. The results indicate the problem of misdiagnosis deserves attention. However, there are several limitations to this review.
As with all literature reviews, publication bias may be an issue and for this topic is inevitable and information on misdiagnosis is often unreported. This review was also limited to reports in English and may have missed reports in other languages. The majority of reports are from Africa and may not be representative of other geographies. Because the review was designed to identify reports of misdiagnosis, it is possible studies reporting errors and quality of HIV testing may have been missed.
Due to both the paucity and heterogeneity of data, it was not possible to conduct more quantitative analyses. Studies included were generally not designed to determine the exact cause or causes of misdiagnoses, a weakness cited across research on diagnostic errors [86].
This review focused on human errors and quality system failures. While we did identify some reports of cross-reactivity [10,22,24,25,27,32,38,56,60], reports did not provide conclusive information on what exactly caused cross-reactivity. Possible biological factors due to antibodies from inter-current infections, adverse environmental exposure to assay components, HIV subtype or shared false cross-reactivity in RDTs within an algorithm may be issues requiring further investigation.
Acute and early infection did not appear to be a significant cause of false negative diagnoses; however, few studies identified reported on acute infection. Retesting among HIV-positive individuals taking ART did emerge as a key factor contributing to a substantial proportion of false negative diagnostic errors and misdiagnoses. Further research is needed to understand how ART, as well as the use of antiretroviral drugs for prevention, for example, preexposure prophylaxis, may impact the performance of HIV RDTs, as well as how frequently people previously diagnosed with HIV and on ART retest.

Conclusions
Our review has identified a number of factors and practices that may contribute to diagnostic error and HIV misdiagnosis. Although no study could fully determine and quantify the exact cause(s) of misdiagnosis, our review elucidated four key factors: (1) suboptimal testing strategies, primarily the use of a tiebreaker testing strategy to rule in HIV infection, (2) user errors including interpretation of weak reactive lines, (3) inadequate management and supervision of testers and (4) retesting among people with HIV on ART. Most, if not all, are avoidable with appropriate guidelines, training and supervision. The consequences of misdiagnoses are serious at an individual and public health level. With the momentum to scale-up HIV diagnosis and linkage to ART, a parallel push to improve the quality of HIV testing services and prevent misdiagnosis is essential. and the U.S. Agency for International Development (USAID), with support from the U.S. President's Emergency Plan for AIDS Relief. The content of this supplement is solely the responsibility of the authors and does not necessarily represent the official views of the USA Government or the World Health Organization.

Disclaimer
The funders of the study had no role in study design, data collection, data analysis, data interpretation or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.