How public health authorities can use pathogen genomics in health protection practice: a consensus-building Delphi study conducted in the United Kingdom

Pathogen sequencing guided understanding of SARS-CoV-2 evolution during the COVID-19 pandemic. Many health systems developed pathogen genomics services to monitor SARS-CoV-2. There are no agreed guidelines about how pathogen genomic information should be used in public health practice. We undertook a modified Delphi study in three rounds to develop expert consensus statements about how genomic information should be used. Our aim was to inform health protection policy, planning and practice. Participants were from organisations that produced or used pathogen genomics information in the United Kingdom. The first round posed questions derived from a rapid literature review. Responses informed statements for the subsequent rounds. Consensus was accepted when 70 % or more of the responses were strongly agree/agree, or 70 % were disagree/strongly disagree on the five-point Likert scale. Consensus was achieved in 26 (96 %) of 27 statements. We grouped the statements into six categories: monitoring the emergence of new variants; understanding the epidemiological context of genomic data; using genomic data in outbreak risk assessment and risk management; prioritising the use of limited sequencing capacity; sequencing service performance; and sequencing service capability. The expert consensus statements will help guide public health authorities and policymakers to integrate pathogen genomics in health protection practice.


INTRODUCTION
The specialist area of pathogen genomics has assumed increasing prominence during the COVID-19 pandemic, and sequencing of SARS-CoV-2 has been performed on a much greater scale than for any previous pathogen [1]. The COVID-19 Genomics UK (COG-UK) Consortium was established rapidly in response to the COVID-19 pandemic. COG-UK leveraged expertise, people and equipment from diverse organisations as part of a single national sequencing programme. Its role in research, surveillance and policy-making has been central to the delivery of a pathogen genomics service and capability for the United Kingdom (UK). The rapid development of consortium tools allowed for harmonised analysis and reporting, and sharing of data and intelligence OPEN ACCESS in the UK and internationally [2]. Sequence data and lineages were made publicly available through GISAID and the COG-UK website [3][4][5].
Sequencing has informed our understanding of the evolution of the SARS-CoV-2 virus. However, there is limited practical guidance for health protection teams in public health authorities about how pathogen genomics data should be used in practice; for example, in terms of risk assessment and risk management of cases, clusters and outbreaks. Policy-makers and service commissioners also need information to support decisions about how to fund pathogen genomics services.
The Delphi consensus-building research method aims to develop expert consensus statements [6]. This method is appropriate where the goal is to allow a group of individuals to take part in 'structured communication' about a complex issue [7], such as developing clinical guidelines. For example, the Delphi method was used in the context of COVID-19 to create expert clinical practice statements on the management of respiratory failure [8]. As part of the Public Health Agency's COG-UK-funded Public health Risk Assessment using Genomic Methods And Tools In Context (PRAGMATIC) Study, we planned and undertook a Delphi study. Our study aimed to develop a set of expert consensus statements that could guide public health authorities about how they should use pathogen sequencing to support health protection risk assessment and outbreak management.

Study design
To identify baseline information to inform the development of our study, in September 2021 we undertook a rapid review of published reports on the use of pathogen sequencing in public health practice. We searched PubMed, using the following keywords and phrases: 'public health genomic sequencing' , 'whole genome sequencing in public health' , and 'sequencing and public health action' . We restricted the results to those published in English between January 2018 and August 2021. We chose not to undertake a formal systematic literature review due to our assessment that it would not be feasible to conduct a systematic review, Delphi study and to implement the findings in our practice during the 6 month funded timeline of our project. We therefore chose to use a modified Delphi method in which we asked the expert respondents to provide free-text responses in the first round of the study to inform the development of the second-round statements [9,10].
The major themes identified in our rapid review highlighted the importance of specific contexts, such as the use of sequencing in identifying variants, the investigation of the effects of transmission, immune escape and severity of disease. It was reported that sequencing played a key role in outbreak investigation within healthcare and residential care settings. The literature described methods of interpretation and dissemination of sequencing results to key stakeholders. These themes formed the basis of our round one questions.

Participant recruitment
Forty-four people were invited by email to participate in the study. The invitees were from institutions associated with the production or use of pathogen genomics information in the UK, including COVID-19 Genomics UK Consortium (COG-UK) members, the four UK public health agencies, healthcare providers and academic researchers. Participants were initially identified through our professional networks, and onwards through a snowballing approach. The participants were from a range of professional backgrounds, including scientists, clinicians, policymakers, researchers, and public health practitioners. Thirty-three potential participants were identified through their roles in the UK's four public health agencies (in health

Impact Statement
Our study will inform public health authorities, commissioners and healthcare providers about how pathogen genomics can be used in practice. The consensus statements developed through the Delphi process provide a framework for organisations to assess their own capability, and to inform the future development and integration of pathogen genomics in practice.
protection or pathogen genomics services), eight were primarily affiliated to academic organisations, and three were affiliated to National Health Service or Health and Social Care Trust laboratories. Nineteen of 33 invitees from public health agencies agreed to participate (58 %), five of eight primarily academic invitees agreed (63 %), and one of three Trust laboratory invitees agreed (33 %). The research team did not participate in the study as respondents. We chose not to collect demographic characteristics of respondents as due to the small number of participants we assessed that it would be possible to identify individuals by the unique combinations of characteristics, which would have undermined the anonymous nature of the study. We asked whether the participant was a producer or user (or both) of pathogen sequence data. If participants missed a round, they were eligible to take part in a later round, as this is thought to reduce the risk of false consensus and to better reflect the views of the invited participants [11].

Delphi process
We invited the participants to take part in three rounds of an online survey, which took place between 5 and 12 January 2022, 1 and 10 March 2022, and 11 March and 22 March 2022. One reminder was sent to all participants during each round. The first round comprised twenty questions with free-text answer fields (Table S1, available in the online version of this article). From this, we developed the second-round statements, which required responses on the Likert Scale from 1 (strongly agree) to 5 (strongly disagree) (Supplementary Materials). Consensus was accepted when 70 % or more of the responses were either strongly agree/agree, or alternatively, 70 % or more of responses were disagree/strongly disagree, on the five-point Likert scale [12][13][14][15]. When consensus was not achieved, statements were revised to improve the clarity or specificity, and were carried through into a subsequent round. The Delphi exercise ended after round three if statements did not reach consensus.

Data analysis and reporting
The expert consensus statements were tabulated and presented alongside explanatory notes informed by the authors' summary of the participants' free-text responses. We followed the Conducting and Reporting of Delphi Studies (CREDES) guidance [6].

Study population
Twenty-five participants (57 %) responded to the initial invite and agreed to participate in the study. Twelve to fourteen of those responded to each survey round. They were a mix of people who produced and used pathogen genomic data (Table 1; Fig. 1).

Delphi round one completion
Twenty questions with a free-text answer field were asked in round one. Fourteen of 25 participants responded. The questions were answered with a range of completeness. Thirteen out of the 20 (65 %) questions were answered by more than ten participants

2.
Public health authorities should use pathogen genomic sequencing as a surveillance tool to monitor geographical spread of variants over time. Strong support was given to the use of sequencing as a tool to monitor the geographical spread of variants after their initial introduction to a population [21].

3.
Public health authorities should sequence enough randomly selected samples to enable unbiased surveillance. Participants highlighted the importance of unbiased surveillance to ensuring that variants are detected as part of routine surveillance. It was emphasised that this is essential in detecting changes in the prevalence of variants and ensuring the variants believed to be extinct are no longer detected [22].
Understanding the epidemiological context of genomic sequence data

4.
The proportion of samples sequenced should reflect the epidemiological context. The absolute number and relative proportion of positive samples sequenced will affect the confidence that can be placed on the results for risk assessment and decision-making. ECDC has published guidelines that pathogen genomics surveillance systems should sequence a minimum of 10 % of positive samples and they discuss the importance of linking this information to local public health systems [23][24][25].

5.
Pathogen genomic information should be linked to contextual epidemiological information to facilitate risk assessment. Sequencing may be used as a tool to support or discount potential transmission events in those identified as having epidemiological links. Epidemiological, demographic and clinical data can supplement genomic data as part of analyses of severity and vaccine escape.
When the aim is to interrupt community transmission, ongoing contact tracing of linked clusters may help prioritise the sequencing of community isolates [26].
Use of genomic data in outbreak risk assessment and risk management 6.
Public health authorities and healthcare providers should use pathogen genomic sequencing in outbreak investigations in health care settings to rule in or rule out transmission events. Much published work has focused on the potential of genomics to identify unrecognised connections between cases, but less attention has been given to the importance of ruling out transmission events, which can effectively disprove transmission between individuals and thus de-escalate a situation. This has recently been addressed with the development of tools like the HOCI Sequence Reporting Tool (SRT) [27]. Tools should contextualise the probability of transmission. It was suggested by participants that maximum likelihood phylogenetic trees should be used, at least at a local level and that further detail should be provided with time-scaled trees, such as those that can be created with IQTREE-2 or BEAST 2 [28,29].

7.
Public health authorities should use pathogen genomic data to de-escalate potential outbreaks that were identified through epidemiological links in closed settings, such as care homes and hospitals. Sequence results have demonstrated the ability to de-escalate outbreaks by excluding nosocomial transmission when multiple external infection introductions have been identified [27,30]. This may facilitate the reallocation of resources away from potential incidents that were shown to be unrelated cases.

8.
Public health authorities should use pathogen genomic data to de-escalate potential outbreaks that were identified through epidemiological links in community outbreaks, such as at public events or functions. When assessing outbreaks in a community setting identifying the source of infection can be very difficult. Pathogen sequencing has helped demonstrate transmission that can often occur in large community settings such as public events [31].

9.
Public health authorities should use pathogen genomic data to inform estimates of epidemic growth. Pathogen genomics is essential in assessing the spread of emerging variants. It has played a key role in the understanding of the effects of mutations on the growth advantage of new variants [32]. 10. Public health authorities should use pathogen genomic data to inform estimates of clinical severity of the disease. During the COVID-19 pandemic, the transition between variants has been accompanied by differences in clinical severity and vaccine effectiveness. Linked pathogen genomic data is important for measuring these factors [33].

11.
Public health authorities should use pathogen genomic data to detect changes in risk of infection in specific settings e.g., schools, care homes etc. Understanding the relative attack rates associated with different lineages or variants in specific setting may help public health authorities assess and manage risk in those settings [27].

12.
Pathogen genomic information should inform evaluation of the effect of vaccines and pharmaceutical therapeutic treatments. In the context of COVID-19, sequence data have supported detection of reductions in vaccine effectiveness associated with the emergence of new variants [34][35][36] and informed booster vaccine schedules [37]. The use of novel COVID-19 therapies has been guided by knowledge of the variant at an individual and population level [38].
13. Pathogen genomic information should inform evaluation of the effect of non-pharmaceutical interventions. The effectiveness of individual actions (such as social distancing and personal protective equipment) and societal restrictions (such as the mandated closure of places of business and education) were affected by SARS-CoV-2 variants, with respect to both the transmissibility and severity. Timely genomic data was important for managing these situations [39][40][41][42].

Continued
Statement no.

Explanation
Prioritising the use of limited sequencing capacity

14.
Public health authorities should prioritise sequencing from outbreaks where it appears there is greater than expected disease severity. Outbreaks with a significantly greater than expected case hospitalisation ratio or case fatality ratio could be an early sign of a variant with greater inherent severity or immune escape, and should result in sequencing to assess this risk [43,44].

15.
Public health authorities should prioritise sequencing of samples from vaccinated people. New variants of SARS-CoV-2 that became dominant after the introduction of the vaccination programme were associated with progressively reduced vaccine effectiveness, possibly due to the selection pressure in the context of high prevalence and high vaccination coverage. Identifying changes in vaccine effectiveness provides opportunities for changes to vaccines or to introduce additional doses to manage the impact of such antigenic changes [43].

16.
Public health authorities should prioritise sequencing of samples from people who have had multiple episodes of infection. Reinfection may indicate immune escape due to the emergence of new variants, and should be monitored using integrated genomic and epidemiological data [45,46].

17.
If it is not possible to sequence all samples, public health authorities should direct sequencing capacity towards outbreak investigations in closed settings, such as care homes and hospitals. Participants emphasised the importance of using genomic information to determine the need for implementing additional protective measures in closed settings during outbreaks [47]. Application of stricter measures may be required in an outbreak of a high-risk variant in a closed setting such as in a care home whereas, an outbreak of the dominant lineage may not warrant the same additional measures. In a hospital however, we see a far more open setting with introductions possible from the community at a more regular interval. In this case we see a greater proportion of spread to wards from outpatient clinics, staff as well as between wards. In this scenario, a more consistent and higher coverage of sequencing may be of use to identify problematic areas and implement IPC measures earlier e.g., improved ventilation, reinforcing mask wearing rules, hand hygiene, etc [48].

18.
If it is not possible to sequence all samples, public health authorities should direct sequencing capacity towards populations in whom new variants might be present e.g., international travellers. When a new variant emerges in one place, there will be a lead time before it reaches other locations. International borders present an opportunity to detect new variants by effectively sampling people who have recent exposure in other regions. Early awareness of new variants can inform the response to their arrival in a timely way [49,50].

19.
If it is not possible to sequence all samples, public health authorities should direct sequencing capacity towards populations in whom new variants might be present e.g., immunocompromised people. The risk of the evolution of new variants may be increased in individuals who are immunocompromised, due to the selection pressure arising from ineffective immune response and consequent longer infection duration. Participants believed that these individuals should be prioritised due to the opportunity to detect new variants in the first person in whom a variant arose and the opportunity to respond to this situation [51]. People who receive treatment with antiviral or antibody therapies, which may create a selection pressure towards resistance, may also be prioritised.

20.
If it is not possible to sequence all samples, public health authorities should direct sequencing capacity towards people who are at greater risk of adverse outcomes from infection. When there is diversity of variants in circulation, information about which variants are responsible for infection in demographic groups that experience severe outcomes may help inform public health risk assessment and management, and wider policy choices aimed at reducing harm [52].

21.
If it is not possible to sequence all samples, public health authorities should direct sequencing capacity towards outbreak investigations in community outbreaks, such as at public events or functions. The ability to identify the transmission pathways may highlight opportunities for future prevention, or alternatively, sequencing may disprove transmission and change the risk assessment of behaviours or events [53]. Sequencing service performance

22.
Timeliness of genomic sequencing results is important to allow results to be acted upon by public health authorities. Timely results enable the assessment of the growth rates of new lineages and can inform advice about mitigation. Rapid genotyping can be a useful adjunct to sequencing for the monitoring of new variants but requires a high level of specificity [54]. Timely genomic results can help inform public health risk management in outbreaks [44].

23.
Clinical teams need timely access to sequence results to inform treatment and infection control decisions. In the early stages of an outbreak of a new variant, sequencing results have been used to inform treatment decisions, which is therefore time-sensitive [55]. Identifying the introduction and transmission of a lineage within a hospital, for example, can reveal breaks in infection control and inform infection prevention and control measures [40,47]. Sequencing service capability Table 4. Continued

24.
Public health authorities should ensure training is provided for health protection, infection control and clinical teams on the interpretation of sequencing results. Interpretation and analysis of pathogen genomics results require a range of multidisciplinary skills. Participants highlighted: data processing and epidemiological analysis; outbreak investigation and management; infection prevention and control methods and factors associated with transmission; knowledge of pathogen biology and evolution; understanding of genomics terms, such as SNPdistance, time to most recent common ancestor (TMRCA), cluster and lineage; ability to interpret phylogenetic trees; understanding of the process and limitations of sequencing; understanding of what sequencing data represents and how it is generated; understanding of the process for selecting samples and the proportion of samples being sequenced at a given; understanding the limitations of SARS-CoV-2 sequencing; the ability to integrate genomic cluster data with epidemiological data at scale; knowledge of infection prevention and control (IPC) practice; understanding of the IPC implications for genomically-linked and genomically-refuted clusters; understanding the limitations of epidemiological information; and understanding the application of genomics in investigating vaccine effectiveness and immune escape.

25.
Public health authorities should use tools such as rapid genotyping for surveillance of lineages. Genotyping methods have supported epidemiological surveillance and public health response, but preparation of relevant assays requires prior knowledge of the haplotypic relationship between specific SNPs and a lineage of interest. It is a compromise of increased speed (and reduced cost) compared to the more detailed genomic information provided by sequencing [53]. Genotyping provides less resolution and cannot be used to create transmission trees, for example.

26.
Public health authorities should use bioinformatics tools to elicit unsuspected transmission events. With the increasing availability and the development of bioinformatics tools like CIVET and outbreaker2 the ability to investigate unsuspected transmission events has become more comprehensive. These tools allow investigators to look beyond the local level of outbreaks and build connections not previously identified [56,57].

27.
Public health authorities should analyse sequencing data in more detail than lineage as part of the investigation of outbreaks. Genomic information at the level of genomic lineage can indicate where transmission has not occurred. It is common, however, when a lineage is dominant, for most infections to be of the same lineage. More detailed analysis can provide further information in outbreak scenarios of the same lineage [57].
Tools that can infer the probability of transmission, like those developed in the HOCI study, should be used to investigate outbreaks [27]. Where possible maximum likelihood phylogenetic trees should be used. Table 4. Continued while seven of the 20 (35 %) questions were answered by fewer than eight participants ( Table 2). The full responses are in Supplementary Materials.

Delphi rounds two and three completion
In round two, a consensus was reached for 18 (69 %) of the 26 statements presented (Summarised in Table 3, full results in Supplementary Materials). In round three, one question did not reach consensus. We concluded that consensus would not be achieved following a further round of questioning for this statement ( Table 3). The Delphi study was concluded after round three.

Statements that achieved consensus
The statements that achieved consensus, with accompanying explanatory notes, are presented in Table 4.

Statements that did not reach consensus
Statements that did not reach consensus after the third Delphi round are presented in Table 5.

Pathogen genomics bioinformatics tools described in expert responses
During the completion of the first round of free-text answers, a series of bioinformatics tools were suggested for analysis of sequence data (Table 6).

DISCUSSION
The COVID-19 pandemic highlighted the need for guidance on the use of pathogen sequencing for policymakers and practitioners. Our study provides expert consensus statements to support investigators and researchers to develop services. The statements should inform the development of systems and processes for delivering and monitoring pathogen genomics services, and inform practical operational use of pathogen genomic information by public health teams. The themes and statements may be used as a self-assessment tool by public health agencies in the development of their capabilities. Delphi studies can also reveal points of professional controversy or disagreement, i.e. dissensus. Our study only resulted in one statement about which there was disagreement, which was about the potential use of sequence data in outbreak investigation where the data did not pass (unspecified) thresholds of sequence depth or quality. This may reflect differing technical knowledge of participants, or true disagreement Table 5. A summary of the consensus statement that did not reach consensus and the rationale provided by the study panel.

1.
Analysts should only include sequences above defined sequence coverage depth when investigating transmission events. (This is an overall quality control value provided for all sequencing results).
In the first iteration of this statement, responses demonstrated a need for greater clarity about the implications of sequencing coverage for interpretation of lineage calls. The statement was twice revised for precision and clarity, and did not reach consensus. Table 6. Pathogen genomics bioinformatics tools described in expert responses
BEAST 2 'BEAST two is a cross-platform programme for Bayesian phylogenetic analysis of molecular sequences. It estimates rooted, timemeasured phylogenies using strict or relaxed molecular clock models. It can be used as a method of reconstructing phylogenies but is also a framework for testing evolutionary hypotheses without conditioning on a single tree topology' [29].
Civet 'Cluster Investigation and Virus Epidemiology Tool civet is a tool developed with 'real-time' genomics in mind. Using a background phylogeny, such as the large phylogeny available through the COG-UK infrastructure on CLIMB, civet will generate a report for a set of sequences of interest i.e., an outbreak investigation' [56].

HOCI Sequence
Reporting Tool (SRT) 'The COG-UK Consortium Hospital-Onset COVID-19 Infections (COG-UK HOCI) study aims to evaluate whether the use of rapid whole-genome sequencing of SARS-CoV-2, supported by a novel probabilistic reporting methodology, can inform infection prevention and control (IPC) practice within NHS hospital settings' [27].
IQTREE-2 'IQTREE-2 is a fast and effective stochastic algorithm to infer phylogenetic trees by maximum likelihood. IQ-TREE compares favourably to RAxML and PhyML in terms of likelihoods with similar computing time' [28].
Transcluster 'Transcluster is an R package for inferring and viewing transmission clusters from sequence alignments and sample dates' [59].
about the technical limits at which data can be interpreted with validity. The qualitative responses from participants in round one revealed information about bioinformatics tools for practical use of pathogen genomics data in public health practice, which may be useful for teams that are developing their capabilities.
Pathogen genomic sequencing is now used as part of the public health management of infection with Mycobacterium tuberculosis, Campylobacter spp., Staphylococcus aureus, Escherichia coli, Shigella spp., Listeria spp. and Salmonella spp. [16,17]. The use of pathogen genomics in public health practice may inform direct clinical care and the effectiveness of outbreak management [18]. The COVID-19 pandemic highlighted the need for public health agencies to have multidisciplinary pathogen genomics capability, supported by effective data infrastructure and processes. Timely genomic results integrated with epidemiological data can support public health teams to assess and manage the risk of outbreaks and wider epidemics in real time. HDR-UK and CLIMB-COVID have supported the integration of COG-UK genomic data with pseudonymised data for research purposes, and genomic data are available through the trusted research environments of the UK nations.
The consensus statements reached in our study reflect a specific point in the management of the COVID-19 pandemic, and the capability, capacity and maturity of UK pathogen genomics services. Though there are generalisable lessons from this work beyond COVID-19 and into the future, as the policy, testing, practice, methods and funding situations evolve over time, the findings may become less relevant. The Delphi method is readily usable in a new situation or context, should a need to update recommendations develop.

Strengths and limitations
As the participants were individuals who created and used pathogen genomics data during the COVID-19 pandemic the findings highlight the real-world, practical issues that they faced in the implementation and application of a pathogen genomics service.
The number of participants was relatively low, and we limited invitations to people who were working within the UK public health and genomics system. The timing of our study coincided with the emergence of the Omicron variant, which doubtless competed for the attention of our invited potential participants. The short windows for completion of the study rounds may have limited the response rate. The Delphi study method inherently incorporates some subjectivity from the participants and the researchers, which we have aimed to mitigate through our transparent adherence to a conduct and reporting standard, and open data sharing with our report [7]. There is no standard method for confirming consensus in Delphi studies [14]. The 70 % threshold for agreement that we used reflects a degree of consensus, about which there is no objective gold standard to measure against. Alternative rules or thresholds could have resulted in variation in our results.
Our study focused on gaining expert consensus from individuals who work in pathogen services, or who directly use the information that those services produce. As such, the study did not have patient, public or service user participants. We did not have any patient or public members of our research team. We believe that it is important that future work should have patient and public representation in their design and conduct.

CONCLUSION
Pathogen genomics capability has been greatly enhanced by the investment and focus that resulted from the COVID-19 pandemic. The expert consensus statements from the PRAGMATIC Delphi study participants will help public health authorities and policymakers plan for the longer-term applications and integration of pathogen genomics in health protection practice.

Conflicts of interest
The authors declare that there are no conflicts of interest.

Ethical statement
We received confirmation from the UK National Research Ethics service that, as the study involved healthcare professionals as participants who were recruited in the capacity of their professional role, the research was exempt from ethical review by NRES Research Ethics Committee under section 2.3.14 of Governance arrangements for Research Ethics Committees.