Knowledge of public patient involvement among health economists in Ireland: a baseline audit [version 1; peer review: 2 approved]

Background: While there is growing public and patient involvement (PPI) in health research in Ireland, there is little information on the extent to which PPI has permeated the field of health economics research. The aim of this scoping study is to explore the knowledge and use of PPI among health economists in Ireland. Methods: This paper is based on a survey questionnaire administered electronically to all members of Health Economics Association of Ireland (HEAI). The study provides a baseline audit on the knowledge and use of PPI in health economics research.  A review of the international literature on the interface between PPI and health economics informed the design of the questions used in the survey. Results: Half of respondents incorporate PPI in their health economics research.  Two thirds report their knowledge of PPI as weak or fair; only 8% say it is excellent. PPI, when it occurs, is largely focused on the identification of research priorities and appropriate outcomes. Just over one third of respondents currently evaluate their PPI activities in health economic research; there is almost no focus on impact as part of the evaluation process. Conclusion: Health economists in Ireland do not have a strong knowledge of PPI, either in practice or in theory. They find PPI challenging due to limited time, resources, information and the absence of appropriate skills to engage patients and the public in their work. Education and training are needed to support greater use of PPI in health economics research.


Introduction
Patient and public involvement (PPI) is increasingly recognised as an essential component of health research. PPI is defined as research carried out 'with' or 'by' members of the public, rather than 'to', 'about' or 'for' them; the term 'public' includes patients, potential patients, carers and people who use health and social care services, as well as organisations that represent people who use these services (INVOLVE, 2009). PPI facilitates patients having a greater role in research prioritisation and research design, which is important given the potential of health research in finding solutions to their specific health needs (Popay et al., 2014). PPI also ensures that members of the public who ultimately pay for research through taxation have some say in how research funding is spent. There is certainly an argument that research funding needs to be more aligned with public and patient priorities to ensure that scarce resources are used efficiently and effectively (Brett et al., 2010;World Health Organisation, 2018).
Ireland is in the process of widening and deepening PPI in health research through the PPI Ignite programme, which is funded by the Health Research Board (Health Research Board, 2018). The Ignite programme aims to promote the use of PPI in health research in Ireland. Specifically, the HRB have awarded grants to five universities to support health and social care researchers therein to involve patients and members of the public in meaningful ways across the full research cycle (Hendrick et al., 2017). Fortunately, Ireland can learn from other countries in relation to optimal approaches to PPI in health research (Kreis et al., 2013;Oliver et al., 2008;Wilson et al., 2015). INVOLVE (2016) have published resources on good practice and approaches to PPI in the United Kingdom, including a Values and Principles Framework. There are also appraisal guidelines and frameworks for assessing the quality of PPI (Boote et al., 2006;Wright et al., 2010), while the GRIPP2 (Guidance for Reporting Involvement of Patients and Public) checklist has been developed to enhance the quality of PPI reporting (Staniszewska et al., 2011).
While more guidance is now available on PPI pathways in health research, there is much less clarity on the role of PPI within health economics research (Goodwin et al., 2018). Health economics research is concerned with issues of efficiency, effectiveness, value and behaviour in the production and consumption of health and healthcare. Health economists use economic reasoning and techniques to examine costs, benefits and consequences arising from the allocation of scarce health care resources. PPI can be used to help inform the economic evaluation of health care programmes and interventions, particularly in relation to what costs to include and what outcomes to capture (Pizzo et al., 2015). Some countries currently require PPI to be used in health technology assessment, but the practice is not widespread (Weeks et al., 2017). An economic evaluation of PPI itself covering costs, outcomes and impact may also be useful to demonstrate the gains from involving patients and the public in the research process. There is little information on the use of PPI in health economics research, but its main purpose currently seems to be to improve data collection (Kandiyali et al., 2018). The proportion of HTA organizations that both conduct and evaluate PPI activities remains unclear (Weeks et al., 2017). The most reported forms of evaluation relate to process and patient satisfaction. There has been very little evaluation on the overall impact of PPI in health economics research.
A better understanding of how economists use, interpret and apply PPI is important in health research, but there are differences between PPI and health economics which mitigate against its use by practitioners. Health economics is very much concerned with priority-setting across competing publicly funded health care programmes, while PPI draws on the practical and lived experience of the individual patient. Reconciling the private and the public is not always an easy task. Moreover, health economics is often perceived as a technically difficult discipline, abstract in its orientation and removed from the more immediate health concerns of the patient. There is an absence of a common language to bring the two sides together in any meaningful way. But if health economists want to really understand the health production process, either for modelling or measurement purposes, there is an onus on them to hear the concerns and insights of patients.
However, prior to any meeting of minds, we need to establish the views of health economists on PPI, including documenting their current PPI experiences and perceptions regarding the application of PPI within health economics research (Kandiyali et al., 2018). The current knowledge base regarding the interface between PPI and health economics in Ireland is non-existent, so any new data will be helpful in facilitating a greater rapprochement between economists, patients and the public. Health economics is now used routinely in all forms of health research, especially in clinical trials. If PPI is absent from economic evaluation of health care programmes, the voice of the patient will be weaker when it comes to resource allocation decision-making and priority-setting in health and social care. However, PPI also needs to be rigorously evaluated to determine if it's having the desired effect on the research process through detailed cost benefit appraisal (Pizzo et al., 2015).

Aim of the study
The aim of this study is to conduct a baseline audit of the use and understanding of PPI among health economists in Ireland. The context for the study is the HRB PPI Ignite award to NUI Galway and the particular emphasis on the evaluation of PPI in that award. Finding out how much health economists know about PPI and use it in their own work will contribute to the development of a PPI evaluation culture in Ireland.

Methods
The main focus of the questionnaire was on audit, with an emphasis on the use and understanding of PPI among health economists in Ireland. The questionnaire was designed to elicit information on: demography; research group/organisation; current knowledge of PPI; current use of PPI; benefits and challenges of using PPI; evaluation frameworks; the role and value of PPI in health research; and the training needs of health economists in regard to the use and evaluation of PPI in research studies. These topics were identified from a detailed literature review on the relationship between PPI and health economics drawn from the following databases: Google Scholar, Medline, EMBASE and CINAHL. The following key words were used in the literature search: public and patient involvement and: health economics; health economists, economics; health technology assessment; economic evaluation; costs; cost effectiveness analysis; outcomes; and impact.
Following the development of a draft questionnaire, we conducted a pilot study, using the online software SurveyMonkey. The pilot was completed by four respondents, three of whom were health economists, while the fourth was an experienced researcher working in PPI at NUI Galway. Following the pilot, changes were made to the wording and sequencing of some questions.
The main survey was then administered to all members of the Health Economists Association of Ireland (HEAI). The vast majority of practising health economists in Ireland are members of HEAI. The Association was set up in 2001 to foster networking and the sharing of information related to health economics on the island of Ireland. The membership at the time of the survey was 135 people, the majority of whom were working in universities, research organisations, industry and the private consultancy sector. The final survey questionnaire was uploaded on SurveyMonkey and email contact was made with all of the members of the HEAI in the first week of July 2018 through the auspices of the Co-ordinator of the Association, Professor Ciaran O'Neill at Queen's University, Belfast.
The Co-ordinator emailed all members of the HEAI incorporating the following message from the authors: Please find attached a brief note of a study on patient public involvement (PPI) in health economics research that we are undertaking at NUI Galway. The link below takes you to the survey we are using to collect the opinions of health economists in Ireland in relation to the work.
The attachment that was sent to members of HEAI described the study as follows:

The Health Economics and Policy Analysis Centre (HEPAC) at NUI Galway, in conjunction with Public Patient Involvement (PPI) Ignite at NUI Galway is undertaking a small scoping study on the use and role of PPI in health economics research in Ireland. PPI is defined as doing research 'with' or 'by' patients and members of the public rather than 'to' or 'about' or 'for' them. It involves consultation, involvement and communication with patients and the public to incorporate their perspective into the research process.
The survey will take approximately 10-15 minutes to complete. We would appreciate that this survey be completed by 30th July 2018.
The research has five objectives: The study will add to an evolving literature on the use of PPI in health economics research.
The online survey on SurveyMonkey opened with an explanation of the purpose of the study, including a reference again to the five objectives. A follow-up reminder to complete the survey was sent to potential respondents on the 24 th July 2018 and the survey closed on the 30 th July 2018. The data was subsequently analysed using Microsoft Excel 2013.

Ethical considerations
The research did not require ethical approval due to the low risk, non-sensitive nature of the study. Participants willingly participated in the study through completing the on-line questionnaire and did so anonymously.

Results
The response rate for the survey was 33% of all HEAI members, divided almost equally between males and females (n=45). Almost half of respondents (47%) are in the 31-40 age bracket ( Figure 1). Only 5% of respondents are aged over 60 years.
Almost 75% of respondents have been working in the health economics field for more than 5 years, with 14% having 20 years or more service ( Figure 2). One third of respondents hold positions as lecturers/professors in an academic institution; researchers constituted 22% of all respondents, while 15% hold dual roles of researcher and manager/director of their organisation.
While half of all respondents are either current or past users of PPI in their health economics research, knowledge of PPI is weak and fragmented across all respondents ( The most common route through which health economists recruited PPI contributors is patient groups at 94%, followed by interest/advocacy groups at 78% (Table 2). Recruitment through carer's groups was 50%, while word of mouth and social media   reached 33% respectively amongst respondents. The least reported approach for PPI recruitment amongst health economists was radio interview and online videos at 17% and 6%, respectively. Table 3 shows the different tasks carried out by patients and the public involved in health economics research. Two thirds of respondents stated that public/patients help to identify research priorities and potential outcomes, as well as helping with sharing/ dissemination of results. PPI plays a significant role in the review of reports (56%) prior to publication. Nearly two fifths of people involved in PPI advise on the instruments used to measure outcomes; a similar proportion support the production of lay summaries of scientific reports. One third of respondents say that PPI supports participant recruitment. There is much less This is followed by patient/public satisfaction at 34%. One quarter of respondents evaluate protocols for meetings and advance information for participants. A further 13% of respondents evaluate the cost of including PPI in their research projects, including costs associated with the recruitment process and attendance at meetings. Only 6% of respondents evaluated the overall impact of PPI. The majority of respondents do not publish or share the results of internal evaluation of their PPI activities. When it does occur, the sharing of evaluation results with participants at meetings is the preferred dissemination method favoured by respondents.
Qualitative reflections on the benefits and challenges of using PPI in health economics research Health economists were also asked to reflect qualitatively on the benefits and challenges of using PPI in their research. The main theme that emerged as a benefit of using PPI in health economics research is that of maximising the relevance of the Carer's groups 50 Social media 33 Word of mouth 33

Radio interviews 17
Online videos 6 Other 6 Table 3. PPI roles and tasks.

Roles and tasks %
Help to identify research priorities 67 Help to identify potential outcomes 67 Help with sharing/ dissemination results 67

Review reports 56
Advise on instruments to measure outcomes 39 Help with producing lay summaries of scientific reports 39 Help with participant recruitment 33 Help with data collection 22 Help with writing reports 11 Help with data analysis 0 Other (please specify) 28 involvement in data collection and writing reports. There is no evidence of any PPI in data analyses.
When asked to rank the reasons why they involve patients and/or members of the public in research activities, more than half of health economists choose to help ensure economic reasoning reflects patient and public values as the primary reason (Table 4). A further 20% reported to conform with funding requirements as the primary reason for using PPI. Helping to promote fairness and inclusion of a range of stakeholders in economics research ranked second highest at 59%, while the need to ensure that research survey materials are understood by the public and patients ranked third highest at 35%.
Approximately 37% of health economists currently evaluate PPI activities in relation to their own research activities. The most evaluated activity reported is the monitoring of process, specifically in relation to tasks and recruitment (63%) ( Table 5). Some of the changes health economists reported that they have made to their research approach as a result of PPI inclusion in their research include the following: • changes to questionnaires and study materials based on the feedback from PPI contributors.
• concentration on what matters most to patients.
• modified approach to economic analysis and evaluation.
The main challenge that emerged from the survey is the difficulty of patient and public recruitment for PPI work. Research has traditionally been conducted on patients not with patients or by patients. Economists have traditionally viewed the public as potential participants in their studies rather than having them otherwise involved in the process. The following quotations illustrate some of the concerns around recruitment: There is risk of fatigue when the same group of representatives are used repeatedly. Given the relatively small population of Ireland, finding patients representative groups for specialised topics can be challenging (R1).
Finding the relevant patients organisations to contribute to the process and finding patients that have same awareness of the health system is a challenge for PPI engagement. Also to find appropriate individuals and ensuring they can contribute beyond personal situation is a major challenge (R9).
The issue of incentives for PPI participants was also highlighted as a challenge by some respondents. In Ireland, people are not always paid to contribute to PPI research activities and therefore may face financial barriers in contributing their time and, in the words of one contributor, may have difficulty remaining committed to the research process from beginning to end. More generally, economists point to the difficulty in realising PPI in instances where little or no funding is available in the original grant for PPI activities. One respondent reported paying for PPI out of other research funds. Ethics can also inhibit a proactive approach to payment when money is available, as some ethics committees are generally wary of financial transfers to people contributing to PPI. Accounting structures within universities can also inhibit or block payment to PPI contributors.
Another challenge highlighted by respondents was the difficulty of presenting the content of economic evaluation in an easy to understand format for PPI purposes. The ability to contribute to the research process can be difficult when there is a lack of technical expertise, especially in regard to economic methodology among patients and the public. While PPI contributors may understand the disease, economists may not always do enough to explain the economic approach to the evaluation of programmes and interventions, according to one respondent. Timing imbalances, conflicts of interest, power imbalances and the tokenistic attitudes of some researchers to patients and the public were other PPI constraints highlighted by respondents.

Discussion
PPI is relatively new in overall health economics research in Ireland. Consequently, it is not surprising that knowledge of PPI amongst health economists in Ireland is weak. A poor knowledge and understanding of PPI implies that its adoption in health economics research is also likely to be weak. If the basics of PPI are not fully understood by health economists then it is difficult to see PPI being used effectively in research programmes. Training is one of the avenues through which the knowledge base of PPI in health economics can be expanded and improved, thereby ensuring greater adoption of PPI in health economics research programmes. The health economists in this study recognised the need for more education and training in relation to PPI, with 94% saying that they would undertake training if it were made available. The main areas of training need identified by respondents were: the recruitment of patients; PPI procedures and protocols; managing conflict of interest; best practice on the remuneration of PPI contributors; and methods of evaluating PPI. Respondents also wanted more information on incorporating PPI feedback into research design and the preparation of lay reports on technical pieces of research. Guidance on how to successfully incorporate PPI into grant funding applications was also referenced.
Only a small number of health economists in Ireland have conducted any evaluation of PPI within their own research activities, mainly in relation to process and protocols. Without knowledge of impact, it is impossible to be definitive about the value of PPI in health research. The challenge is to ensure that more health economists include PPI in their work and in their evaluation methodologies. It should be easier for economists to undertake evaluation of PPI, given the methodologies at their disposal, especially in relation to instruments for the measurement of costs and benefits. NICE have recommended that organizations should develop an evaluation plan at the same time as the PPI activities are established, incorporating patients or members of the public on the evaluation team.
The most difficult part, however, is to assess the relationship between PPI and health impact, as mediated through practice and policy change arising from the research process. Any change can have a long time horizon and it can be difficult to attribute research activity directly to changes in health outcomes.
There are still many barriers to PPI, some of which are referenced in this study. Health economists report PPI to be a challenging task due to limited time, inadequate resources, knowledge deficiencies as to the meaning of involvement and the absence of skills to conduct effective PPI. The evidence is that funding bodies do not always view PPI as a high priority and consequently do not provide enough funding to cover PPI a ctivities (Blackburn et al., 2018;Gove et al., 2018). The absence of funding can undermine the potential of PPI, particularly in economics research where the philosophy of PPI is not yet rooted (Pizzo et al., 2015).
Health economists also reported that a lack of commitment and burnout on the part of PPI contributors can be a challenge for researchers. The population and size of Ireland means that the same cohort of PPI contributors are often recruited for different PPI projects, leading to a major sustainability challenge. The overuse of the same people in PPI can lead to participation fatigue, as well as predictability in relation to the involvement. PPI must reflect the variation in patient and public experience, if it is to be relevant to in decision-making and prioritysetting.
The tokenistic attitude of some researchers is also a major challenge for PPI adoption, often resulting in PPI contributors feeling unimportant, undervalued and unable to contribute fully to the overall PPI process. Underlying and extended data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
The data is available to view upon free registration with the repository (DANS EASY); once registration is complete, intending users can view the data with their log in details.

Grant information
The author(s) declared that no grants were involved in supporting this work. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Trinity College Dublin, Dublin, Ireland
This is an important contribution towards driving for good multidisciplinary research that includes both health economics and public and patient involvement (PPI), and to ensure that these two interlinked elements become mandatory whilst spending the public's money on research. The Survey method, administered through Health Economists Association of Ireland (HEAI) provides an accurate cross-section of economists working in health across Ireland and the results provide an important reading of current practice. The conclusions are useful to ensuring that any health economist training in Ireland develops a good understand of PPI to ensure good quality research design. The authors also address 'tockenistic attitudes' to PPI which is an important take home for the research community.

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Yes Are the conclusions drawn adequately supported by the results? Yes