Involving citizens in priority setting for public health research: Implementation in infection research

Abstract Background Public sources fund the majority of UK infection research, but citizens currently have no formal role in resource allocation. To explore the feasibility and willingness of citizens to engage in strategic decision making, we developed and tested a practical tool to capture public priorities for research. Method A scenario including six infection themes for funding was developed to assess citizen priorities for research funding. This was tested over two days at a university public festival. Votes were cast anonymously along with rationale for selection. The scenario was then implemented during a three‐hour focus group exploring views on engagement in strategic decisions and in‐depth evaluation of the tool. Results 188/491(38%) prioritized funding research into drug‐resistant infections followed by emerging infections(18%). Results were similar between both days. Focus groups contained a total of 20 citizens with an equal gender split, range of ethnicities and ages ranging from 18 to >70 years. The tool was perceived as clear with participants able to make informed comparisons. Rationale for funding choices provided by voters and focus group participants are grouped into three major themes: (i) Information processing; (ii) Knowledge of the problem; (iii) Responsibility; and a unique theme within the focus groups (iv) The potential role of citizens in decision making. Divergent perceptions of relevance and confidence of “non‐experts” as decision makers were expressed. Conclusion Voting scenarios can be used to collect, en‐masse, citizens' choices and rationale for research priorities. Ensuring adequate levels of citizen information and confidence is important to allow deployment in other formats.


| INTRODUCTION
Between 1997 and 2010 £1.4 billion of public funds were invested in infection research in the United Kingdom (UK) 1 comprising the major source of funding (54% of the overall funding). Despite the majority of funding coming from public sources, UK citizens currently have no formal role in determining where or how this money is invested within this field of research.
The Health Research Authority (HRA) in the UK has set out clear aims and objectives for involving patients and the public in the research process, setting out a clear mandate for improving patient and public involvement (PPI) directly in research. 2 Within the field of infection, research organizations and funders have engaged widely with PPI at the levels of "direct care" and "organizational, design and governance". 3 However, PPI is less present in "policy" aspects of infection research as described by the framework for patient and family engagement in health and health care proposed by Carman and colleagues. 3 There is also a lack of tested approaches for setting research priorities for use by public agencies who make funding decisions. 3 In this sense, PPI at the policy level seems to stop at informing patients and the public about research but does not partner them in the decision process of setting priorities for research, and hence a long way from citizen-led priority setting. 4 Despite a growing body of literature exploring citizen-led priority setting, [5][6][7][8][9] there is little evidence to support the feasibility of large-scale interventions to collect views of citizen priorities for infection research.
Greater understanding of how to facilitate such an intervention and the potential challenges and biases associated with this is yet to be defined.
To test the feasibility of partnering citizens in priority setting, there first of all needs to be a clear and validated method for informing and involving a large number of citizens in the decision process.
This must take into account many of the common issues associated with researcher engagement with patients and the public and sharing decision making across research and health care. [10][11][12][13] An important contextual factor for any PPI activity is the health literacy of the population; in the UK for example, approximately 43% of citizens require assistance in understanding written health information. 10,14,15 This means that as well as ensuring that issues of access and participation are addressed on a logistical level, involvement at the strategic level runs the risk of tokenistic or technocratic involvement, if methods are not fit for purpose. [16][17][18] Citizens must have time and space to express their information needs and levels of confidence when asked to participate in decision-making processes.
We set out to develop a tool and test its feasibility as part of a reproducible methodology capable of exploring citizen priorities for infection research from a large population sample, with the potential to inform priorities across research programmes.

| Scenario design
A case scenario (Appendix S1) was created as a means of accessing a large, sample of citizens' views on their priorities for research funding in the field of infection. The scenario was created in consultation with a number of researchers with experience in PPI and health literacy, acting as a working group for this project (TMR, EC, ECS, LSPM, AH and RA). The scenario was then critiqued by the unit's patient representative who has experience of developing decision support tools and evaluating patient facing interventions. She provided structured feedback on the dimensions of internal validity, consistency of presented information, and assuring bias was not introduced in the way that the working group presented the information. Following revisions, the scenario was then piloted on two non-medical support staff within the research department, a junior doctor, and five citizens through social networks not involved in health-care delivery or research. The aims of the scenario were to put citizens visiting a university open day event in the position of a large global charity, with the means to allocate funding to one or two research areas within the theme of infection. During this scenario, the citizens would be presented with information on six infection research areas or "finalists" from within this theme and would be given the opportunity to allocate funding to one or two of six research areas. Funding was selected at 2x £50 million (equal to 1 vote each) or 1x £100 million (equal to 2 votes) allocated to two or one of the finalists, respectively. Citizens were also required to write a brief justification for funding allocation decisions on the back of each cheque that they wrote.
After development of the scenario, the working group developed a poster to implement the scenario at the university public festival.
The six "finalists" in the scenario were selected following a review influenza-pandemic potential.

| Scenario implementation & analysis
The scenario was set up and tested across 2 days on 7th and 8th May 2016 at Imperial College London university public festival, which was visited by over 15 000 members of the public. A stand was set up in one of the festival buildings within the infection zone, which had research presented on all six research areas within our scenario. One researcher (TMR or EC or RA) manned the scenario over a two-day period, with short vignettes provided to guide what information could be given to individuals who sought further clarification during participation in the scenario.
Following each day of the festival, all cast votes and justifications for why individuals allocated funding were collated and analysed using NVIVO pro11.0 software, to analyse voting habits and the major drivers of voting choices. Voting practices and justifications were compared across both days to assess them for reproducibility.

| Focus group evaluation
Twenty citizens (recruited through Cherry Picked, UK-a specialist qualitative recruitment company) were invited to participate in a 1hour in-depth focus group on May 16th. Citizens were recruited using specific screening criteria, from a database of 20 000 individuals from around the UK who had signed up with the recruitment agency previously. The screening criteria, which included information such a demographics and location, as well availability during the days/hours that the workshop was to be held, identified a sample of 500 individuals.
An initial email was sent to all 500 individuals, advertising the workshops. The respondents were then stratified according to recruitment criteria, and 20 individuals were selected for inclusion. For this evaluation, we aimed for an equal mix of genders, educational statuses and age groups as well as a diverse mix of ethnicities. Citizens must not have attended the university public festival. Two further emails were sent to these individuals confirming their attendance and sending directions to the venue. The focus groups aimed to explore the participants' views on PPI in research policy decisions, to explore methods for collecting citizen views on research priorities, and evaluate the method and results from our pilot of the scenario at the public festival.
One researcher (ECS) led three separate focus groups, following a topic guide (Appendix S2). A second researcher (TMR) observed the focus groups, making notes on the session to allow areas of researcher reflexivity to be considered and addressed during data analysis. The questions were developed by five researchers highly experienced in qualitative research but from diverse health and social science backgrounds (TMR, ECS, EC, LSPM and RA) and were then pre-tested with a patient representative with experience of qualitative research (having previously co-authored with the research team). As with the scenario development, these were then piloted on two non-medical support staff within the research department, a junior doctor, and five citizens through social networks, not involved in health-care delivery or research. The questions were open ended and included prompts. In the focus groups, space was provided for individual perceptions and opinions to be aired. There were no normative statements of what the public should or should not do-instead the participants were asked to give their views on whether the public should be involved at all in the decision making, how they should be involved, and if the tool provided was clear and suitable for use with members of the public.
All participants were consented, and discussions were audio recorded and transcribed verbatim. These were then analysed using a mixed inductive and deductive technique, using NVIVO pro11.0 software. For inductive analysis, one researcher (TMR) reviewed all transcripts allowing initial codes to be generated by line by line coding for first order codes, and a second researcher (LSPM) independently coded transcripts. 21,22 During line by line coding, the comments provided by the observer were considered with the aim of complementing and balancing areas of reflexivity derived from the analysts' own background, beliefs and prior experiences. 23 After comparing the coded transcripts, a list of emerging categories were developed in addition to the codes generated through analysis of the public festival voting. After meeting and agreeing on key categories and themes within the text, two researchers (TMR & RA) independently preceded to systematically crossreview the text, coding passages based on these agreed codes and categories, subsequently grouping them into overarching themes. On review, any discrepancies were discussed within the working group, including our patient member (FH), and consensus reached. Examples of key opinions and ideas from the text for each main theme identified were then charted to allow mapping and interpretation of the results. 21 This project was reviewed by the regional ethics committee, who deemed that citizen interviews during focus groups and festival involvement (anonymous voting and written justifications) did not require ethical approval and was allowed under the local clinical governance and research compliance office oversight. Written consent was obtained from the focus group participants. Participants providing anonymous votes and written justifications at the festival were not required to provide written consent. day 2) over both days (Figure 1). Table 1 provides a breakdown of categories and themes that arose from analysis of these justifications for citizen voting habits.

| University public festival voting
Three themes emerged from the categories reported on the voting slips. These were (i) information processing (I), which included the relevance of the infection to the individual, its geographical proximity and also how information has been presented to those individuals; (ii) knowledge of the problem (K), which includes our current understanding of the problem, what solutions/treatments are available, and areas where greater understanding/research is required; and (iii) responsibility (R), which includes the individuals person feeling of responsibility to act, feelings of the need to help others, and responsibility to future citizens and public health in general.  to the responsibility that lay with the decision maker (individual citizen)

| In-depth evaluation of the intervention
in making big choices about where funding should be allocated. Whilst there was agreement that greater transparency and clarity on where money is spent on research was needed, the participants struggled to agree on the overall role that citizens should have in informing final decisions on allocation of resources. Here, over half of the citizens described feeling that the final decisions on allocation of resources should be trusted to those who they saw as "experts" in the field, and thus could weight up all of the information available to make informed decisions about the effectiveness of investment in that area. Other participants disagreed however, and felt citizens ought to have a right to say where research funding, to which they may have contributed, is to be spent. There was consensus that citizens should know the outcomes of decision-making processes, regardless of level of citizen involvement in the process.

| DISCUSSION
Using a simplified, scenario-based voting intervention we were able The discussion regarding relevance of citizens being asked to make such strategic decisions arose at the focus groups but did not surface at the university open day where citizens were asked to cast votes.
This difference may be important in informing the appropriateness of voting-based methods versus group-based analysis for collecting citizen priorities and preferences for large-scale strategic decisions.
Furthermore, the reported reaction and rejection to being asked to assume responsibility for priority setting reported within our focus groups are a well-described phenomenon in the literature. 25  Recently, van Bekkum and colleagues described the technocratic approach taken by research bodies in the UK to PPI in decision making. 18 Here, they describe how funding bodies tend towards selecting citizens' with technical knowledge and expertise in the field that they are involved, which can lead to a narrowing of the focus of PPI in making decisions surrounding funding opportunities. One of the leading reasons for this was due to the practical challenges posed by involving large numbers of citizens in the decision-making process. 18 Another was the due to the challenges that members of the public can face when being entered into the scientific environment as a citizen representative. 18 These were themes common to our reported results, where despite reporting interest in involvement in decision making our own participants also voiced concerns over "non-experts" ability to make decisions based on what they perceived as limited information. Therefore, as well as providing a new simplified avenue to promote wider citizen engagement in strategic decisions this type of tool may also be positively adapted to demonstrate that broader members of the public can make informed decisions about priorities for research that are congruent with current funding strategies. This can be demonstrated with recent investment in the field of drug-resistant infections and emerging infections, such as Ebola and Zika virus, which based on the information provided in our scenario citizens also prioritized as important avenues for funding to be allocated currently.

| Limitations
There were several limitations to our study. Firstly, the voting was undertaken at a university public festival which may have biased voting given that people attending where likely to have a higher than average educational background. Furthermore, the stand was situated in a building with exhibits on all six infection themes that were included for voting, meaning that citizens may have been influenced by further information provided within the building. Voting was also anonymous meaning that we were unable to collect demographic information on those casting votes. This is something that we aim to address in future evaluation of the tool. Finally, focus group work identified the potential biasing effect of placing the infection themes upon a world map, as individuals may have been more likely to vote for themes with closer proximity to themselves. This may have influenced some of the voting towards drug resistance infections. However, the majority of the six infection themes were placed in geographically relevant locations to where they are currently major issues, meaning that this influence may have been justified if proximity to oneself was a deciding factor for the individual. Further work is now underway to explore alternative methods for displaying this type of information.
Moreover, knowing that 43% of the population are health literate must also be taken into account when refining such tools and the manner in which they are implemented. Improving health literacy remains a priority when thinking about individuals and decisions about their own health, but also as it has implications for the extended roles citizens are being asked to take. Citizens indirectly make decisions which impact on public spending for health and research and overseas development. The tool tested here focused on a specific area of strategic decision making and the level of information that the public were able to assess and assimilate. Our tool development was informed by the health literacy gap. There is further potential for learning by implementing the tool with specific groups with specific/ known health literacy needs.

| CONCLUSION
This study demonstrates that this tool provides a useful means of engaging members of the public and also can be used to gauge public confidence in casting votes. The tool may be implemented either in a group setting with discussion, or by allowing space for respondents to state their level of confidence or additional information needs. Voting must be taken in context as a useful addition to the portfolio of current methods for promoting wider citizen engagement in strategic decision making.