Optimizing patient and public involvement (PPI): Identifying its “essential” and “desirable” principles using a systematic review and modified Delphi methodology

Abstract Background There is international interest in the active involvement of patients and the public. However, consensus on how best to optimize its application is currently unavailable. Objective To identify and assess the underlying principles of patient and public involvement (PPI) in health and social care services, research, education and regulation across medicine, dentistry and nursing. Design A four‐phase methodology: (i) an extensive systematic review of published and grey literature; (ii) inductive thematic analysis of review findings; (iii) development of best practice principles; and (iv) consensus testing of identified principles using a modified Delphi methodology. Setting and participants Twelve systematic reviews and 88 grey literature publications were reviewed leading to the unique identification of 13 principles later assessed by 18 PPI experts. Results Essential consensus (>75% agreement) was obtained for nine principles reviewed. Working in equal partnership and sharing information achieved the highest consensus rates: 16/17 essential 94.1%; 1/17 desirable 5.8%. The four remaining principles that failed to reach essential consensus were categorized as desirable by expert respondents. No principles were considered irrelevant. No alternatives were suggested. Discussion Expert respondents suggest essential principles must be achieved to optimize PPI best practice. To advance PPI practice, desirable principles should also be aspired to wherever possible. Conclusions This study's innovative approach advances existing knowledge by providing previously unavailable consensus about PPI best practice. Research findings hold important theoretical and practical implications for educators, regulators, researchers and practitioners looking to effectively work together.


| INTRODUCTION
The practice of patient and public involvement (PPI) is multidisciplinary in nature and international in scope. [1][2][3] Although other definitions are acknowledged, for the purposes of this study, PPI is defined as the "active involvement of patients, service users, carers or family members in activities done with or by, rather than to" them. 4 Often influenced by mandatory policies or requirements, 5,6 PPI has been applied to a variety of settings including health-care services, 7 regulation, 8 education 3 and research. 1 However, in spite of its widespread application and well-documented benefits including enhanced safety and quality of care, [9][10][11] consensus on how best to optimize its application is currently unavailable.
One explanation for this discrepancy is a lack of a common language to share PPI practice. 3 For example, in both policy and practice, the term PPI is not universal in its application or definition. 12,13 Such issues have arguably led to an overwhelming number of circular debates held primarily amongst academics with limited patient involvement, 13 causing the fundamental principles required for individuals to effectively work together (the central tenet of all PPI initiatives regardless of its "label," context or topic of interest) to be largely overlooked. 13 Whilst discussions about the semantic intricacies of PPI are important in certain contexts, as argued by Brett et al 11 it is equally important to identify principles that can best enhance its potential impact. 8,11,14,15 Given that PPI now forms a central tenet of most policy requirements in health and social care services, it is imperative that consensus is achieved on the principles underpinning it. In spite of international interest, there is as yet limited research that explores PPI best practice, and less still that has been rigorously assessed or evaluated from a patient perspective. 16 This research, therefore, aims to inform PPI best practice across historically fragmented disciplines through the innovative application of a systematic review and modified Delphi methodology. The research is unique not only in its scope and methodology, but also in its inclusion of both an evidence-informed and patient/public expert perspective. It contributes to, and advances existing knowledge by exploring areas of consensus and conflict, the central premise of Delphi methodologies, across the inter-related but until now disparate settings of: health and social care services, education, research and regulation across medicine, dentistry and nursing.

| METHODS
This research draws on a four-phase methodology: (i) an extensive evidence review of published and grey literature exploring PPI across the settings of health and social care services, research, education and regulation in medicine, nursing and dentistry; (ii) inductive thematic analysis of review findings; (iii) development of PPI principles based on thematic analysis findings; and (iv) consensus testing of identified principles using a modified Delphi technique comprised of three rounds. Similar to the original, the modified Delphi technique begins with a pre-defined set of carefully selected items. 17 For the purposes of this review, principles were determined by the literature and not from discussions held with the expert panel, although their opinions and suggested modifications were sought in free text boxes during all iterative rounds. 18 The primary advantages of a modified Delphi technique include enhanced response rates, and a strong grounding in extant literature, 17 thus warranting its inclusion in the proposed research.

| Phase 1: Evidence review
To ensure transparency of findings and rigour, review findings follow the Centre for Reviews and Dissemination 19 and National Collaborating Centre for Methods and Tools guidance. 20 The peer review proposal is registered in the PROSPERO database No CRD42016035415.

| Search strategy
To ensure sufficient coverage and be inclusive of the most up-to-date information, both peer-reviewed and grey literatures were reviewed.
The search terms listed below designed to maximize sensitivity and specificity were developed using the SPICE framework 21

| Peer review
To avoid duplication of existing work, a systematic review of reviews published between 2010 and 2016 was conducted to assess how PPI is initiated and developed across the aforementioned settings. As advised by an information specialist (GT), one author searched Medline, EMBASE and PsycInfo for articles published in the English language.
Due to limited resources, the authors could not ensure a sensitive interpretation of non-English articles and these were therefore excluded in both search strategies to maintain research integrity. Reference list and citation searches of eligible studies were also conducted. The search terms used to review peer-literature were as follows: "service users" OR "patient*" OR public OR lay AND involvement OR participation OR engagement OR co-production AND review.

| Grey literature
For the purpose of this research, grey literature was defined as "that which is produced on all levels of governmental, academic, business and industry in print and electronic formats but which is not controlled by commercial publishers." 23 It was included on the basis that it would help validate the results of published literature searches, identify the most up-to-date information 24 and respond to criticisms of existing literature operating in silos. 13 A conventional search engine (Google) was used to identify grey literature using the following search terms: patient OR public OR "service user" AND involvement OR co-production OR participation OR engagement AND tool OR guide OR kit. Review searches were conducted over a period of five days (01/03/2016 -05/03/2016).

| Peer review
Studies were selected through a two stage process. Two reviewers independently examined abstracts of identified studies for study inclusion using pre-defined inclusion criteria: systematic reviews published in the English language between 2010 and 2016; exploring PPI in medicine, dentistry or nursing across one or more of research, regulation, health-care services and educational settings. When an inclusion decision could not be made from the title and abstract alone, the full article was retrieved. Accepted studies were then read in full and independently assessed for study inclusion. Any discrepancies that could not be resolved by discussion were sent to a third reviewer for clarification until consensus was achieved.

| Data extraction
Two reviewers independently undertook data extraction using a piloted data extraction form. Information extracted included: year F I G U R E 1 Peer review selection process published; literature location, aim and design; study population and sample methodology; intervention type; and outcome findings.

| Quality appraisal
Although not the focus of the review, included reviews were quality appraised independently using the Revised Assessment of Multiple Systematic Reviews (R-AMSTRA) measurement tool to ensure some reliability of the conclusions drawn. No formal quality assessment of the grey literature was undertaken as this was not the primary focus of the review.

| Phase 2: Thematic analysis
Review findings were thematically analysed using an inductive approach. 25 One researcher inductively analysed all content with a random 10% of documents analysed by a second researcher to ensure reliability. 25 No discrepancies or modifications were made to thematic analysis findings during this stage.

| Phase 3: Development and synthesis of PPI principles
Principles were inferred from the general themes identified in phase two. Inferred principles were developed based on content similarity and presence across the four areas of interest (research, regulation, education and health-care services). Principles that did not appear in all domains of interest were not developed further. Expert respondents were invited to suggest additional principles not identified and revise the structure and language of identified principles to ensure accessibility and understanding. Suggested modifications were made accordingly.

| Phase 4: Delphi technique
Defined as "a widely used and accepted method for achieving convergence of opinion concerning real-world knowledge", 26 26,29 Its main benefits include the attainment of anonymous "expert" opinions without the time and geographical restraints experienced in alternative methods. 28 However, since its inception, the Delphi has undergone a series of modifications with no universally agreed guidelines surrounding its appropriate design, "expert" definition or appropriate expert panel size. 15,18 Despite this, it is widely accepted that the appropriate use of a Delphi methodology requires a high degree of methodological precision and research rigour. 29 For the purpose of this research, experts were defined as "those who are knowledgeable about, or skilful in, a particular area." 30 In this instance those with experience of PPI in health and social care services, research, education and/or regulation. Expert respondents were asked to: Round 1: rank order principles in terms of importance based on their own experience to establish preliminary priorities amongst them.
Round 2: clarify or revise their judgement following a revised questionnaire built around the responses generated from Round 1.
Round 3: make any final comments or revisions following the sharing of ratings, and identification of minority, and majority opinions.
Results from rounds one and two were fed back to the experts in the form of line graphs and simple textual summaries of statistical data. The order in which identified principles were presented to expert respondents remained consistent. The possibility of creating an order effect is acknowledged. 31 All rounds were administered via email or posted if requested.

| Sampling
Patient and public involvement experts identified through the social and professional networks of the research team were invited to take part through advertisements distributed through the Patients Association, Lived Experience Network and Developers of User and Carer Involvement in Education network.
In line with accepted guidance, 27 which suggests recruiting 10-18 participants, 22 experts were contacted to take part to allow for potential dropouts or non-inclusion.

| Consensus definition
Consensus was defined as 75% or greater agreement. This is based on the results of a recent systematic review where percentage agreement was identified as the most common definition of consensus.
75% was the median threshold used to define consensus in a random selection of 100 studies. 32  were included. One review was UK centric, the remaining 11 drew on international literature predominantly from the US, Canada, Australia and New Zealand. The overall quality of included reviews was mixed.

| Principle identification
Thematic analysis of review findings led to the identification of 13 principles listed in Table 1. These principles were then tested for consensus amongst an expert population.

| Respondents
A total of 21 experts responded, with three incomplete responses provided. Eighteen responses were therefore included for analysis. Respondents identified themselves as: service users or carers (n = 6); researchers (n = 4); lecturers or professors in nursing and/or midwifery (n = 3); public and patient manager/officer (n = 2); patient educator tutor (n = 1); director of teaching and learning (n = 1); and interprofessional education co-ordinator (n = 1).

| Round 1
No consensus was achieved at any rank order level when expert respondents were asked to rank order principles based on their own experience ( Figure 3). The rank order value of two for working in equal partnership achieved the highest agreement rate of 55.5% (n = 10/18). All other rank orderings were highly variable.
Reflecting the ethos of co-production and feedback from expert respondents regarding the difficulty of this exercise, a collaborative decision was made to order principles into three clearly defined categories: those deemed necessary; desirable; or irrelevant. This decision was made on the basis of difficulties experienced in rank ordering principles and not the total number of principles included.

| Round 2
Of the original 18 respondents, 17 agreed to take part in the second

| Round 3
During round 3, a total of seven categorization alterations were made (listen, assess and respond n = 1 re-categorization; acknowledge n = 1; tailor n = 2; evaluate n = 1; resource and invest n = 2), leading to the re-categorization of two principles as essential  No alternative principles or suggested modifications were proposed by expert respondents at any stage.

| DISCUSSION
This research advances current understanding of PPI practice by identifying and evaluating its essential and desirable principles from both a The lack of consensus reported in the first Delphi round is perhaps to be expected, given the complexity of PPI reported in existing literature. 11 In particular, available evidence highlights the influence different contexts have on the appropriate forms and functions of PPI. 14 For example, those working in nursing may rank principles differently from those working in dentistry because they have different agendas to address. In acknowledgement of this, a collaborative decision was made by both the research team and the expert panel to categorize proposed principles into three distinct categories: essential, desirable, or irrelevant.
Inevitably, principles that achieved majority consensus were often those that provided solutions to the most frequently cited barriers of effective PPI. 14,15,33 For example, one frequently identified barrier is the unequal power dynamics caused by the paternalistic relations that often exist between clinicians and patients. 33 The principle of working in equal partnerships identified in the review is a direct solution to this. Other examples of this parallel relationship include listening, assessing and responding to the information shared to prevent a sense of abandonment or lack of change, a further problematic experience of ineffective PPI. 14,15,33 Principles that failed to achieve "essential" consensus and were therefore categorized as "desirable" are described by expert respondents as principles that can "transcend" mere compliance to PPI agendas. One expert respondent wrote "those [desirable principles] aspects may not always be achieved hence they can only be desirable. By comparison, those I considered essential are attainable." This distinction between the desire for all principles to be achieved, and the reality of achieving them is worth noting, as PPI is contingent on available resources, suitable conditions and buy-in from all those involved. 14,15,33 One important distinction to make is that the principles suggested provide quality guidelines for best practice, not prescriptive rules. The proposal of a "one size fits all" approach to PPI would be inappropriate, as no single PPI initiative will work for all situations, individuals or agendas. However, our research suggests that whilst PPI must be adaptable to local circumstances and objectives, the essential principles required to underpin its effectiveness may well be universal in their application.

| Strengths and limitations
Strengths of this research include its application of a recognized systematic review process 19 and inclusion of a large sample of both peerreviewed and grey literatures. Grey literature is often excluded from systematic reviews due to its associated searching difficulties. 24 Expert respondents were also purposively sampled volunteers, and the potential for bias in this approach is acknowledged. 18 However, given the requirement for experts to have subject knowledge, expert respondents could not be selected at random. 18 Hasson et al suggest the involvement of individuals with subject knowledge may enhance the content validity of Delphi findings, and the use of successive rounds may also increase concurrent validity. 29 The validity of these results should not therefore be undermined. Finally, response exhaustion can be an issue when using the Delphi method. 18 Although one respondent did not participate in rounds 2 and 3, the response rate of 94.1% achieved in this research is highly favourable.

| Implications for practice
This research holds important practical and theoretical implications for policy makers, patients, carers and professionals wishing to work together across the continuum of health and social care. Principles that are considered essential must be present for PPI to be effective.
Principles identified as desirable must also be aspired to if PPI practice is to "transcend" mere lip service to mandatory PPI agendas. From a theoretical perspective, this research is the first of its kind to bridge the disparate corpus of PPI knowledge across a multitude of disciplines and topic areas leading to the co-production of a principle taxonomy pertinent to all those looking to work together. Snow, an inspirational woman in every way.

CONFLICTS OF INTEREST
No conflicts of interest have been declared.