Search strategy

The medical literature was searched for relevant studies in PUBMED; the detailed search string is included in the supplementary material. Personal libraries of TB MAC members were searched for modeling papers (free text search for ‘model’), and mathematical modeling journals were scanned for any papers describing TB models. Within this set of retrieved citations, potentially relevant records were identified through a text search for ‘diagn’ OR ‘test’ OR ‘screen’. Citations of all selected original articles [6] published after the 1^st of January 2000 to March 1^st 2013, were reviewed. The database of relevant citations is regularly updated by TB MAC and available online at: http://www.tb-mac.org/Resources/.

Inclusion and exclusion criteria

Papers were eligible for inclusion if they were written in English and published after the 1^st of January 2000 to restrict to recent diagnostic strategies with relevance to current policy decisions. Papers were excluded if the model focused on diagnosing latent TB infection, or evaluated only diagnostic tools that fell within the existing standard of care at the time of publication, based on our understanding of TB diagnostic practices. Therefore this review was focused on modelling novel diagnostic tools or algorithms for active TB that are NOT currently in use as the standard of care. For the purposes of this review, the Xpert MTB/RIF test (Xpert, Cepheid Inc., CA, USA) was not yet considered as standard of care in any setting. The selected papers were independently assessed by two reviewers (AZ and RMGJH) for inclusion and all included studies underwent double data extraction by the same two reviewers; disagreements were resolved through consensus. In defining ‘mathematical model’ we followed Garnett et al [7] and included decision analytic, transmission, operational or within-host models, but excluded purely statistical models and studies using models to only estimate resource requirements without corresponding measures of health-related outcomes [8].

Papers were grouped into three subject areas that were not mutually exclusive: 1) population level impact/transmission, defined as models including a transmission component or measuring epidemiological impact at the population level; 2) health systems, defined as models including compartments that represent points of interactions between patient and health care providers or institutions; and 3) model based cost-effectiveness and cost-utility analyses, following the definitions by Drummond et al. [8]. To further clarify these definitions, transmission modeling generally assesses the population level impact of interventions in terms of changes in TB incidence, TB prevalence or TB mortality over time and can estimate the potential future benefit of introducing a novel diagnostic intervention on those population-level outcomes. Health systems models, which may include transmission and economic evaluations, involve explicitly modeling the interaction between patients and the health system or health care provider. These models are important for evaluating TB diagnostic interventions in the local context and can improve our understanding of operational elements such as patient and diagnostic delay or other key elements of the patient–provider interaction that may influence a test's potential impact and costs. We followed the definitions of Drummond et al. for cost-effectiveness and cost-utility analyses, including any type of model (transmission, health system, cohort, etc.) that includes a comparison of two or more alternative strategies both in terms of costs and consequences (efficacy or effectiveness) [8]. Economic evaluations can rely on many different types of models, but economic evaluations are defined by the study purpose or question they are answering (cost-effectiveness, budget impact, etc.), using different types of models to answer different economic questions. Decision analysis and Markov models are the most common approach for evaluating cost-effectiveness of novel diagnostics; however cost components can also be added to transmission or health systems models.

Data extraction and synthesis

For all papers information was extracted pertaining to the research question and result. Extracted data included: details of the population and diagnostics pathways that were explored, model methodology (model type, sensitivity analyses, details of model structure) and, where appropriate, details on how the cost-effectiveness analysis was conducted, including questions about the costing perspective employed and scope of the costing parameters. A complete summary of all data extracted is available in the supplementary material. The results were discussed by a broad group of experts during the TB MAC meeting on “Impact and cost-effectiveness of current and future diagnostics for TB” in Amsterdam, April 24–25^th 2013 (www.tb-mac.org/WorkAreas/WorkArea/4).

Is enough modeling being done?

Since the approval of Xpert in 2010, a sharp increase in TB diagnostic modeling has been seen both among papers modeling Xpert and among papers evaluating alternative nucleic acid amplification tests (NAATs). Compared with only 8 TB modeling papers published from 2000–2007 inclusive (approximately one per year), 10 TB diagnostic modeling papers were published in 2012, 6 of which focused on Xpert.

The 36 included papers were grouped into the three non-exclusive categories: 16 papers included population impact/transmission modeling, 5 were health systems models, and 24 included cost-effectiveness models. Novel tests evaluated across the 35 papers included: several NAATs, light-emitting diode (LED) microscopy, Determine TB-LAM (Alere Inc., MA, USA) a lateral-flow urine lipoarabinomannan detection assay, FastPlaque TB and FastPlaque-Response (Biotec Labs Ltd, UK), serological tests including Anda TB (Anda Biologicals, Strasbourg), MTT assay, a non-commercial colorimetric assay (ICN Biomedicals) and sputum processing methods including bleach sedimentation and sample dilution. A table of novel TB diagnostic tests included in the evaluated studies with brief descriptions is included in Table 1. Novel diagnostic strategies included the expansion of culture, drug sensitivity testing (DST), chest x-ray and mass miniature radiography (MMR). Several hypothetical diagnostic tests, scenarios or algorithms were also evaluated including: hypothetical NAATs with improved speed, sensitivity and specificity; a dipstick style test; rapid DST; tests resulting in reduction in diagnostic delay; and rapid point of care with improved sensitivity, specificity and reduced time to results compared to smear microscopy or conventional drug susceptibility testing (Tables S1.2, S2.2 & S3.3 in Data S1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Novel diagnostic tests for active TB disease evaluated in included modeling review.

https://doi.org/10.1371/journal.pone.0110558.t001

The majority of studies (83%, 30/36) modelled high TB incidence settings; 39% (14/36) of evaluated studies modelled the South African setting specifically, reflecting the large burden of TB in South Africa and the recent move and investments towards novel technologies in improving TB control. Only 6/36 (17%) studies -all cost-effectiveness analyses- modelled low TB incidence settings including the United States, United Kingdom, and Finland. While the majority of studies are modelling settings with the highest TB burden, many important epidemiological settings (e.g., China, Southeast Asia outside of India) have few or no models to provide insight into potential cost-effectiveness or epidemiological impact of novel diagnostics or diagnostic strategies. As a result decisions with important repercussions for the TB epidemic and TB control budgets are frequently being made in the absence of supporting insight that models could provide. As tests and diagnostic algorithms are developed and evaluated, models are needed that represent a wide range of diagnostic tests, strategies and settings to help support data-driven policy decisions.

What questions are modelling studies trying to answer?

Models covered a wide range of objectives in evaluating novel diagnostics. Transmission and health system models investigated the impact of diagnostic delay (n = 5), and demonstrated several key insights including a threshold for average delay to diagnosis beyond which an epidemic will escalate [9], the importance of applying novel diagnostics in both private and public health systems to reduce diagnostic delay [10], and that test sensitivity was a key determinant of diagnostic delay [11]. In prison populations it was shown screening at entry and annually with MMR could keep TB prevalence below 1% [12]. In the case of drug-resistant TB, it was demonstrated that current strategies with long delays to diagnosis will probably not halt the spread of MDR-TB, and reducing the delay through improved access to DST and second-line treatment may reduce transmission of drug-resistant TB [13], [14]. Transmission models highlighted the importance of considering different elements of the diagnostic pathway (n = 4) including the pre-diagnostic infectious period; others linked operational and transmission models, demonstrating that operational context elements such as patient diagnostic delay, access to care and loss to follow-up can inform setting specific models useful for data-driven policy decisions [15]–[18].

In terms of economic evaluations, models provided comparisons of the cost-effectiveness and impact on TB incidence and mortality of various novel or hypothetical diagnostics and strategies (n = 24) [16], [19]–[41]. Many studies demonstrated the cost-effectiveness of novel diagnostics, often highlighting the settings and implementation strategies and/or algorithms in which those tests were most cost-effective. Among cost-effectiveness models, the most frequently studied diagnostic assays included Xpert MTB/RIF (n = 7) and alternative NAATs (n = 4). Most (n = 16) of the cost-effectiveness models compared new assays to an existing baseline and generated an incremental cost-effectiveness ratio (ICER), of which 69% (n = 11) concluded that novel TB diagnostics were likely to be cost-effective relative to that baseline. Some technologies including NAATs implemented in low TB prevalence settings were judged not to be cost-effective [27],[30], and serology was shown to be more costly and less effective than existing diagnostics such as sputum smear microscopy [28]. Cost-effectiveness studies also highlighted barriers or challenges associated with roll-out of novel diagnostic tests including operational barriers or increased indirect costs, associated with HIV or multi-drug resistance (MDR) care [19], [25]. A transmission study demonstrated that DST for all retreatment cases could be highly cost-effective [40]. Evaluations of hypothetical point of care (POC) tests suggested that a highly specific, low cost POC test would be highly cost-effective, with the greatest impact in settings with poor infrastructure [37].

Do models include the relevant features?

In each of the three model categories we evaluated relevant features associated with modelling methods, structure and parameters including: outcomes of interest, type of model, sensitivity and uncertainty analyses, costing methods and data sources, how drug susceptibility and HIV status were incorporated in the model along with false negatives and positives.

Is uncertainty properly captured?

Sensitivity analyses were performed in 14/16 transmission studies (87.5%), yet only 6 considered variation in two or more variables at a time [12], [15], [17], [19], [40], [43] (Table S1.3 in Data S1). Half of the papers calibrated their models to existing real data (e.g. incidence trends or point estimates). Among the 16 transmission studies evaluated, 13 (81%) explicitly accounted for the impact of transmission before diagnosis. The impact of individuals with false negative diagnoses (i.e. patients with active TB who were misdiagnosed as TB negative and therefore continued to transmit) was estimated in 6/16 studies (37.5%) [10], [11], [16], [17], [19], [43], of which 5 assumed that these patients would re-enter into the same diagnostic pathway at the same rate as ‘pre-diagnostic’ individuals with active TB. Three health systems studies (60%) performed sensitivity analyses [13], [16], [17], only one of which considered more than one variable at a time [17].

All economic studies performed sensitivity analyses, and 13/24 (54%) assessed sensitivity to more than one variable at a time [19], [20], [22], [23], [26]–[28], [33]–[37], [39], [40] (Table S3.4 in Data S1). Fourteen of 24 (58%) explicitly modelled false positives [16], [19], [22], [24], [26]–[28], [30], [31], [33], [35], [39], [41] (which may have an impact on costs as patients then receive unnecessary treatment and follow-up), 14/24 (58%) studies explicitly modelled false negatives [16], [19], [20], [22]–[24], [26], [28], [30], [31], [34], [36], [39], [41] (which may have an impact on health outcomes as they remain undiagnosed, untreated and potentially infectious), Only three studies allowed false negatives to re-enter the diagnostic pathway, in two cases entering the same diagnostic pathway as those entering for the first time [16], [26] and in the third a second pathway for false negatives was modelled [39].

Sensitivity analyses were frequently limited to one- or two- way analyses. More recent models increasingly include multi-way sensitivity analyses as well; however, thorough investigation and discussion of both uncertainty in model structures and parameter values (both point estimates and associated ranges) remains a largely unmet need. More transparent approaches to uncertainty may help to build understanding and trust in modelling approaches among policy makers.

Most models evaluated do not generate predictions that can be validated against data collected in the future; therefore this systematic review is limited in its ability to assess whether model predictions could be validated in real life. This review is also limited by its ability to review only English language publications.