A systematic review of economic evaluations of whole-genome sequencing for the surveillance of bacterial pathogens

Whole-genome sequencing (WGS) has unparalleled ability to distinguish between bacteria, with many public health applications. The generation and analysis of WGS data require significant financial investment. We describe a systematic review summarizing economic analyses of genomic surveillance of bacterial pathogens, reviewing the evidence for economic viability. The protocol was registered on PROSPERO (CRD42021289030). Six databases were searched on 8 November 2021 using terms related to ‘WGS’, ‘population surveillance’ and ‘economic analysis’. Quality was assessed with the Drummond–Jefferson checklist. Following data extraction, a narrative synthesis approach was taken. Six hundred and eighty-one articles were identified, of which 49 proceeded to full-text screening, with 9 selected for inclusion. All had been published since 2019. Heterogeneity was high. Five studies assessed WGS for hospital surveillance and four analysed foodborne pathogens. Four were cost–benefit analyses, one was a cost–utility analysis, one was a cost-effectiveness analysis, one was a combined cost-effectiveness and cost–utility analysis, one combined cost-effectiveness and cost–benefit analyses and one was a partial analysis. All studies supported the use of WGS as a surveillance tool on economic grounds. The available evidence supports the use of WGS for pathogen surveillance but is limited by marked heterogeneity. Further work should include analysis relevant to low- and middle-income countries and should use real-world effectiveness data.

WGS for the surveillance of bacterial pathogens has increasingly been applied in a variety of settings, including for public health [5], food safety [6] and hospital infection prevention and control [7]. Surveillance may be targeted at specific pathogens of interest (e.g. the foodborne pathogen Salmonella enterica), pathogens with specific resistance profiles (e.g. healthcare-associated multidrug-resistant Enterobacterales) or those implicated in an outbreak. With a few exceptions, WGS surveillance is more commonly employed as a second-line investigation, or to a subset of isolates in the form of sentinel sampling, rather than a sequence-first comprehensive screening system [5].
Another barrier to its implementation is that analyses of economic impact are limited and heterogeneous, and the economic advantage to WGS surveillance is unproven [8]. Economic evaluation is an umbrella term for analyses that consider both the costs and consequences (or benefits) of an intervention and a comparator, although partial evaluations may consider only components thereof (e.g. a costing study, which considers costs but not consequences) [9]. The type of analysis depends on whether it is deemed possible or appropriate to monetize all consequences considered, known as cost-benefit analysis, or whether some parameters cannot be monetized. Non-monetized parameters could be considered in natural units (e.g. blood pressure measurement or progression-free survival) known as cost-effectiveness analysis, or in a standardized composite score such as a quality adjusted life years (QALYs), known as cost-utility analysis [10].
This review aims to comprehensively summarize and review available evidence relating to the economic implications of the use of WGS in the surveillance of bacterial pathogens, following a systematic methodology and reporting framework. Of particular interest was the potential application to antimicrobial-resistant pathogens.

Development of search strategy and identification of relevant articles
The PROSPERO International Prospective Database of Systematic Reviews [11] was searched to identify any in-progress review(s) examining the same topic. The research question and search strategy were proposed and refined in discussion with experts in the field (S.P., N.F., J.L.). The draft protocol was reviewed by two experts in the field of WGS and AMR surveillance (N.F., S.P.) prior to registration on PROSPERO (registration number CRD42021289030). There was no specific funding for this review.
To search for available evidence on the economic evaluation of WGS for the surveillance of bacterial pathogens, six databases [Pubmed, Scopus, EconLit, Cochrane Library, NHS Economic Evaluation Database (NHSEED) and BioRxiv/MedRxiv] were searched on 8 November 2021. Reference lists and articles suggested by experts in the field were also screened for inclusion. Search terms were adapted to the requirements of the database being used (see Table S1). Terms included: 'WGS' , 'population surveillance' and 'economic analysis' .
Inclusion criteria were: published manuscripts or pre-print literature in English, available in full text between 1 October 1991 and 1 October 2021 with any form of full or partial economic evaluation of WGS for surveillance of one or more bacterial genera and/or species of World Health Organization (WHO)-defined priority pathogens for research and development of new antibiotics (Table S2) [12]. Studies were included whether or not the threshold of drug resistance was met because the cost of sequencing a bacterial genome does not differ with presence or absence of resistance genes, hence the rationale for this criterion. The date range was chosen to span 20 years and encapsulate all relevant studies, as the first complete bacterial genome was published in 1995 [13]. Duplicate studies, those that did not report an economic analysis and those that did not include surveillance for at least one of the priority pathogen species were excluded, due to our focus on antimicrobial resistance. Reviews and other forms of literature not representing primary analyses were not included in the review, although these were considered for background context. Titles and/or abstracts were screened against inclusion criteria by one reviewer (V.P.). Articles selected for full-text review

Impact Statement
The coronavirus disease 2019  pandemic demonstrated the feasibility and value of large-scale WGS surveillance of severe acute respiratory syndrome coronavirus 2 (SARS CoV-2) to rapidly guide control measures. The upscaling of WGS infrastructure and analytical expertise presents an exciting opportunity to apply WGS-based surveillance to the antimicrobial resistance (AMR) crisis, however the cost-effectiveness of employing WGS for this purpose is underexplored. Our systematic review identified nine economic analyses of WGS for surveillance of bacterial pathogens relating to use in food safety and in hospital infection prevention and control. All of these studies supported the use of WGS as a surveillance tool on economic grounds. They also emphasized the incremental benefit from earlier generation of WGS-based surveillance data to facilitate the timely implementation of -and thus maximize the benefit from -control strategies informed by the resulting data. Our findings emphasize the value of WGS-based surveillance of bacterial pathogens, and lend support to the economic case for implementation of programmes aimed at upscaling sequencing capacity for surveillance of AMR.
were exported to Rayyan [14] and were assessed against inclusion criteria inclusion by two reviewers working independently (V.P. and L.N.), with disagreements resolved through discussion.

Data extraction and quality assessment
Data were extracted from each included study by one reviewer (V.P.) into an Microsoft Excel spreadsheet (Redmond, WA, USA), and checked by another (L.N.). Study characteristics [publication year, year of data collection, economic analysis type, country setting, viewpoint, target organism(s), surveillance application, reporting currency, comparator, WGS post per isolate, comparator cost per isolate], methodological details and outcome data were extracted: estimated or actual impact of WGS on burden of illness (accepting any study definition of burden of illness, i.e. cases or deaths averted); the costs and cost savings of WGS programmes; and the results of any break-even analysis. Given the heterogeneity of studies and reporting, a pilot process was used to refine other data extracted, focusing on the methodology as well as the results of trials. The Drummond-Jefferson checklist, developed to improve the clarity of reporting for economic analyses of healthcare interventions [15], was used as an objective measure of quality. The checklist was completed for each study by two reviewers (V.P., L.N.) independently, with disagreements resolved by consensus.

Data synthesis and reporting
The heterogeneity of economic analysis types, geopolitical contexts, surveillance scales and time points and the limited number of manuscripts precluded formal meta-analysis, so a narrative approach was taken to the synthesis of the methodology and results of the included studies following the recommendations of the Synthesis Without Meta-analysis (SWiM) reporting guideline [16]. Completed PRISMA and SWiM checklists are included in Tables S5-7 [17]. Not all of the extracted outcomes were investigated by all of the studies, so results are presented in a table allowing visualization of heterogeneity. Costs and cost savings are reported in 2020 United States Dollars to enable comparisons, with costs in the currency and year of reporting also shown throughout. Conversions used the average annual exchange rate for the year of reporting according to the Bank of England database [18], and adjusted for inflation to 2020 values using the United States Gross Domestic Product Deflator from The World Bank [19].
In lieu of meta-analysis, a vote count on direction of effect is included. Studies were judged to favour WGS over the comparator where (a) benefits outweighed costs in a cost-benefit analysis, (b) dominance was established in cost-effectiveness analysis and (c) author judgement of realistic case numbers averted in break-even analysis. Costs of illness are presented but not aggregated because frequency of infections due to different pathogens were not reported in all studies.
Heterogeneity is explored through presentation of a method and results table comparing the differing approaches of different studies and the diversity in reporting outcomes. As this review incorporates a variety of forms of economic analysis across a small number of studies, they are not prioritized and are reported in reverse chronological and alphabetical order wherever they appear.

Results of search
Six hundred and eighty-one studies were generated by the search strategy ( Fig. 1, Table S3). Following title and abstract screening, 57 were identified for full-text assessment, of which 8 were duplicates and 49 proceeded to full-text screening. Screening of reference lists from included articles yielded one further article meeting the inclusion criteria. Nine studies were selected for inclusion in this review, with the remainder being excluded for the following reasons: did not include economic analysis (n=22); evaluation of organisms other than bacteria (n=21); review article not presenting primary analysis (n=21) (Fig. 1). In some cases, more than one exclusion criterion applied. A list of excluded studies is included in Table S4.

Characteristics of included studies
The characteristics of the included studies are summarized in Table 1. Of the nine included studies six [20][21][22][23][24][25] were published in 2021, with the remaining three [26,27] published in 2020 and 2019 [28]. The report by Alleweldt et al. includes case report analysis of eight reference laboratories, of which the five conducting foodborne bacterial pathogen surveillance were considered, while the remaining three were not, as they reported on viruses and so were not relevant for bacterial AMR [21]. Five studies analysed WGS surveillance for hospital Infection Prevention and Control (IPC), and four studies examined the role of WGS surveillance for foodborne pathogens. All studies reporting on foodborne pathogens considered salmonellosis specifically, with other foodborne pathogens also considered by Alleweldt et al. [21] and Brown et al. [22]. Of the studies reporting on IPC, those by Gordon et al. [25] and Kumar et al. [29] reported on WGS surveillance of a variety of pathogens of concern within a hospital, while the remaining studies focused on specific pathogens of interest: methicillin-resistant Staphylococcus aureus (MRSA) [26], carbapenem-resistant Acinetobacter baumannii [23] and carbapenemase-producing Escherichia coli [27].
Four studies were cost-benefit analyses, one was a cost-utility analysis, one was a cost-effectiveness analysis, one combined costeffectiveness and cost-utility analyses, one combined cost-effectiveness and cost-benefit analyses and one was a partial analysis  Table S4.
( Table 1). Studies were based on data from high-income countries, with the exception in Alleweldt et al. which included a case study from Argentina [21]; currently classed as an upper middle-income nation [30].
Where stated, the most common comparator for the laboratory bacterial typing analysis component was pulsed-field gel electrophoresis (PFGE), used by three of the four laboratories looking at foodborne pathogens. In hospital studies, the comparator was deemed to be standard IPC practice, a narrative term incorporating epidemiological linkage, and in some cases reactionary typing.

Results of economic analyses: non-monetized
The monetized and non-monetized economic analysis outcomes are summarized in Table 2. For the five studies that considered WGS surveillance in hospitals, all reported on either cases (colonization and/or infection) and/or deaths averted. In the three studies estimating deaths averted due to WGS in an individual hospital or hospital trust, estimates ranged from two (MRSA deaths averted) [26] to six (MDR E. coli, increased virulence scenario) [27]. Gordon [23]. The only other study using QALY as a metric was Jain et al.; however, as they only reported QALY and DALY information for baseline data and not for WGS modelling, no conclusions can be drawn [28].
For the analyses of foodborne pathogens, detailed data on cases averted were not presented. Instead, these were considered in aggregate, in the form of cost savings by all except Brown et al. who reported illnesses averted by pathogen across the 5 years of their programme: Listeria 210, E. coli 5592 and Salmonella 19 792 (total 25 595 95 % CI 9619-43 589) [22].
As most studies report higher laboratory costs with WGS compared to comparators, the benefits derive from the ability of WGS to avert cases of illness. This is achieved through providing actionable data of a higher acuity or in a faster time frame, enabling effective intervention to prevent further infections. These data do not lend themselves to collation, as the epidemiology of the considered infections, the time horizons and the economic impacts of preventing infection differ.

WGS costs
Estimates relating to the cost of WGS per isolate were reported in eight of the nine studies (summarized in Table 1). Alleweldt et al. [21] report these separately for all five included laboratories, while for Jain et al. [28] the estimate is only included in the discussion and is for consumable costs alone. The estimates range from $72.13 by Kumar et al. (USA) [29] to $470.37 in the Italian case study from Alleweldt et al. [21], with a mean of cost of $194.46. Alleweldt et al. [21] provide the most detailed breakdown of the costs considered in relation to providing a cost per isolate for WGS, and in addition to equipment, consumables and staffing they also explicitly consider equipment use and batch size as a factor. Considering this more complete methodology, it may be considered that their estimates of per-sample costs, ranging from $148.31-470.37, are more robust than the other included studies. There was no apparent temporal pattern of cost per isolate becoming cheaper over time, likely reflecting the short publication time span, and cost drivers could not be identified.

Results of economic analyses: monetized
The heterogeneity in the contexts, studies and reporting make comparisons of total costs and costs averted impossible. Net savings at the level of individual hospitals or hospital trusts ranged from 9348 [29] to $977 997 [26]. At the hospital scale, Kumar et al. [29] found WGS to be less costly and more effective than standard care in their base case. However, it should be noted that the estimated sequencing costs quoted were the lowest of all studies included at $72.13 per isolate, within which they did not detail whether additional equipment or staffing costs were considered. Dymond et al. [26] identified that WGS was more effective and less costly than current practice. This was, however, based on a high efficacy of 90 % case reduction in the base case and >30 % in the sensitivity analysis. Considering these limitations, the results should be reviewed with caution. Elliott et al. estimated cost savings of between 51 706 to $64 598 with environmental metagenomics in addition to WGS surveillance in a modelled CRAB outbreak, but did not include a scenario for WGS surveillance alone [23], limiting comparison with the other studies. Net savings were $8.72 million in the first year of screening for six drug-resistant organisms across Queensland, Australia [25], and $539 697 for a modelled outbreak of MRD E. coli for a single Australian hospital [27].
In the studies evaluating foodborne pathogens, the cost per case of salmonellosis was estimated in two studies: $1131 in Australia [24] and $14 072-15 743 across a variety of settings reported by Alleweldt et al. [21] These studies also included a break-even analysis. Alleweldt et al. [21], combining analysis from five laboratories, estimated that between 1.2 and 82.3 cases (CI 0.2-1.1 %) of salmonellosis, or a single salmonellosis-related death, would need to be prevented to break even, while Ford et al. estimated that 275 cases (90 % CI −55-775) or 1.9 % (90 % CI −0.4-5.4 %) of disease would need to be prevented [24]. Although Brown et al. [22] did not report a specific break-even analysis for Salmonella, they did estimate an overall break-even point for all studied pathogens in year 2 of their programme. Furthermore, the authors state that the break-even figures modelled by Alleweldt et al. [21] and Ford et al. [24] are likely to be achievable within the early stages of WGS surveillance implementation, according to their model of salmonellosis cases averted.
Assessing foodborne pathogen surveillance, Brown et al. [22] report on cost-benefit analysis using their actual data since implementation of routine WGS surveillance, while Ford et al. [24] and Jain et al. [28] used modelled scenarios. In view of the superior power of WGS over traditional epidemiology in the detection of outbreaks [31], we believe that the use of observed actual data from real-world application of WGS is likely to be more reliable than modelled estimates. Brown et al. [22] report $503 million in savings by 2019, by which time their WGS surveillance programme had 5 years of maturity. Ford et al. [24] modelled different outbreak scenarios, concluding that for point source outbreaks, WGS would be more costly and no more effective, on the basis that intervention is not possible to avert cases in this type of outbreak. In more prolonged outbreaks, savings of between $44 114-246 100 were predicted, dependent on the type of outbreak and the point at which WGS would allow implementation of effective intervention. Jain et al. 's model [28] concluded that WGS was less costly than standard practice and therefore only reported in terms of net benefits. However, their model did not account for any increase in costs with WGS and relied on expert opinion, mentioned only in the discussion, that WGS consumables would be less costly than the comparator. The Jain et al. model [28] does not account for the non-consumable costs associated with WGS, including technology and human resources, and is not reflective of the findings of the other included studies, where the per-isolate cost of WGS was generally more expensive than comparators.
The methodological approaches of the studies were diverse (summarized in Table 3).

Vote count on direction of effect
In view of the lack of a unifying outcome measure reported by the studies, a vote count on the direction of effect was used to enable comparison. All included studies favoured the use of WGS over comparators. A summary of the results of studies considered in terms of illnesses averted, costs averted and any break-even analysis is presented in Table 2.

Assessment of quality with Drummond-Jefferson checklist
The complete quality assessment using the Drummond-Jefferson checklist across the three areas of study design, data collection, and analysis and interpretation of results is presented in Table 4. All studies scored highly on criteria relating to the study design and justification. There was a high level of transparency in establishing and reporting currency and price data, providing details of models used and establishing benefits. Establishing and reporting primary outcomes, providing an answer to the study question, and conclusions following from the data were also presented well, and in most cases, limitations were appropriately discussed.
Effectiveness estimates were not reported by all studies, and in some cases assumed values were used. Where effectiveness estimates were used, sources were generally not described in detail or critiqued. Quantities of resource use were generally not reported. Productivity changes and their relevance to the study question were generally poorly reported. Most studies did not discuss adjustments for inflation or discounting, although this may in part be accounted for by the short time horizons used. Data were generally not presented in a disaggregated form.
The costing study by Ford et al. [24] represents a partial rather than full economic evaluation, which is reflected in the checklist. The checklist is intended to guide a narrative assessment of the quality of studies rather than prescribe cut-off values to judge a study as high, medium, or low quality. Overall, the studies were considered to represent high-quality works, with relatively minor variation in quality between them.

DISCuSSIon
This review found nine economic analyses of WGS for bacterial pathogen surveillance, of which five evaluated hospital surveillance and four evaluated foodborne pathogen surveillance. The available evidence for the potential economic benefit of whole-genome sequencing in AMR pathogen surveillance is heterogenous and of varying completeness, but broadly suggests that WGS can be economically viable from the public health perspective of foodborne illnesses, and at the smaller scale of hospital IPC. We found that costs for a single WGS test ranged from 72.13 to $470.37. Over the short timescale of   [32].
Most identified studies demonstrated cost savings due to WGS that were largely attributed to averted cases of infection. For this benefit to be realized maximally, WGS needs to be employed early in the analytical pipeline. Conversely, delay in the use of WGS reduces the benefits, as early detection of outbreaks enables timely implementation of interventions to interrupt transmission. Overall, effectiveness estimates were not always used in analysis, or were assumed values. Future economic analyses of WGS should increasingly be able to use effectiveness measures from actual WGS surveillance programmes, rather than relying on assumed or modelled values.
The five studies that looked specifically at AMR pathogens all focused on hospitals and used historic data to model the impact of WGS. The use of real-life outbreak data inherently underestimates the role of WGS, as it limits analysis to outbreaks detected by standard practice, which is less discriminatory than WGS and cannot take account of the potential of WGS to detect outbreaks that are currently evading detection. Future studies assessing the impact of WGS on hospital IPC should use actual data on the effectiveness of WGS surveillance rather than using historic outbreak data derived by conventional means.
No studies were identified that economically evaluated national or regional WGS-based surveillance specifically for AMRpathogens, though such surveillance programmes are being implemented worldwide. The surveillance of foodborne pathogens did not focus specifically on resistant pathogens, but the methodology for modelling outbreaks has clear potential to be replicated for AMR. Future studies are needed focusing specifically on the application of WGS surveillance to AMR.
Modelling the impact of adding additional WGS data into the National Center for Biotechnology Information (NCBI) database by Brown et al. shows the potential for international benefits from WGS surveillance [22]. These authors refer to this as a 'global food shield' , taking a One Health perspective that could be highly valuable in source tracking. The addition of isolate data to publicly accessible databases also presents the opportunity to benefit research and industry in ways not measured by the included studies. In this work, the large contribution of averted listeriosis cases to the economic benefits, despite low case numbers, demonstrates the significant bearing that severity of illness and mortality rate have. This is important to consider in determining the pathogens for which WGS surveillance can be most effectively utilized.  Almost all of the settings evaluated were high income, but the burden of both foodborne illness and AMR is higher in low and middle-income countries (LMIC), particularly in regions of Africa [4,33]. Understanding the costs and benefits of WGS surveillance in LMIC settings could have important implications not only for the health of these populations, but also for global efforts to tackle infectious illness and in particular AMR [34], as highlighted in the WHO Global Genomic Surveillance Strategy [35].
The most detailed analysis of the costs of WGS surveillance was in Alleweldt et al. which was the only study to report in detail the costs of equipment, consumables and resources, and equipment use, and to consider returns of scale [21]. As sequencing capacity and expertise increases, the costs of sequencing are likely to fall, reflecting competition between suppliers, increased automation and economies of scale [36]. That a fall in sequencing costs was not seen in this review is likely indicative of the short timescale during which all included studies were published. Overall, studies were of high quality, as assessed by the Drummond-Jefferson checklist criteria.
This review has several limitations. Inclusion and exclusion criteria were intentionally broad as a paucity of relevant literature was predicted. This resulted in marked heterogeneity, limiting the ability to make direct comparisons or aggregate results, precluding meta-analysis. However, in view of the inherent differences in economic relationships across different time points and geopolitical contexts, it is likely that meta-analysis to determine a 'true effect' would be inappropriate, even if highly comparable outcomes were reported. For example, in considering foodborne pathogens, a host of factors, including climate, aetiology and healthcare services may account for different outcomes in different regions. The differences in study design and methodology also make comparisons of quality assessment challenging, and this has been highlighted previously [37]. The decision to focus on pathogens within the WHO list of priority pathogens for research and development of new antibiotics means that some important pathogens for which WGS surveillance is utilized were excluded, notably Mycobacterium tuberculosis (TB). An analysis of the economic impact of routine WGS with molecular diagnostics for TB in the UK estimated an incremental net benefit of GBP 16.6 million (24.17 million in 2020 USD) over a 10-year horizon [38]. The decision not to include TB in this review was taken in view of the significant biological and genetic differences between mycobacterial species and the bacteria more commonly causing foodborne or hospital-associated infections. While a decision to perform narrative synthesis was made a priori, the form of synthesis was determined post-hoc, with potential for resulting inadvertent bias. It is possible that economic analyses of WGS surveillance showing negative implications may be less likely to be published, and we were unable to assess for this potential publication bias in this review.
That all articles in this review were published since 2019 -and most during 2021 -speaks to the growing interest in the economics of using WGS surveillance on a large scale. Evaluating the economics of genomics for AMR surveillance has been identified as a priority area for action by the Surveillance and Epidemiology of Drug Resistant Infections Consortium (SEDRIC) genomics working group [38]. It is likely that the evidence base will continue to grow rapidly over the coming years, as a sequence-first paradigm is adopted across more real-world settings. It will be important to see a diverse range of surveillance uses, pathogens and geopolitical contexts represented, to understand how this powerful resource can best be used to the benefit of public health. At this point in time, the available evidence paints a positive view of the economic feasibility of large-scale WGS surveillance.