Good to know – This is PPIE! Development of a training tool for public and patient involvement and engagement in pediatric oncological research

Abstract Background Public and Patient Involvement and Engagement (PPIE) in research is still a poorly understood and infrequently practiced concept, although the literature stresses clear benefits for quality of care and research as well as patient satisfaction and empowerment. Aim The presently described project aimed at using different PPIE methods to evaluate the current state of knowledge about and attitude toward PPIE in research among different stakeholders of pediatric oncology in Europe. Based on the findings a tailored training tool directed toward the different stakeholders will be designed. Methods and Results An interdisciplinary steering group developed a mixed‐method 3‐stage process to (1) investigate the current knowledge and attitudes about PPIE using a Europe‐wide cross‐sectional online survey directed toward health care professionals (n = 134) and the patient group (patients, survivors, family members, …) (n = 168). The results were analyzed quantitatively, focusing on group comparisons (t‐tests, X 2 tests). (2) In a live workshop with n = 36 participants (HCPs and patient group) dual moderation teams (HCPs and patient experts) guided the exploration of effective ways for practicing PPIE. Despite classifying PPIE as relevant, both HCPs and patients indicated a low level of knowledge about the concept and terminology (patients: t(334) = −2.82, p = .004; HCPs: t(270) = −2.88, p = .004). While HCPs assumed to already be involving patients in many research areas, this was not perceived by the patient group (X 2 (1, N = 304) = 42.70, p < .001). HCPs and patients named similar obstacles for implementing PPIE in research, though numerous creative solutions were found during the workshop (engagement). (3) The outcomes were integrated into a training tool (White‐Board movie). Conclusion Although HCPs and patients acknowledge the benefit of PPIE, the presented results highlight the lack of awareness about the concept, and the need for effective tools for researchers to integrate PPIE throughout the entire research process, thereby contributing to a sustainable change within the scientific culture.


| INTRODUCTION
Closing the gap between patients, researchers, and health care professionals (HCP) has been linked to better quality of care; higher real-life relevance and translatability of studies; more successful recruitment for clinical trials; and more informed decisions as well as empowerment of those affected. [1][2][3][4][5][6][7][8][9] To date, however, research priorities, decisions about funding allocation, and the conduct of scientific studies is predominantly guided by scientific researchers rather than the affected patient community. 10,11 Furthermore, information on current research projects as well as the results of clinical trials are often only accessible within the academic context and not presented in a language appropriate for lays. 12 To counter this disparity, the UK National Institute for Health Research (NHIR) developed the concept of "Patient and Public Involvement and Engagement in Research" (PPIE). Defined as research "carried out 'with' or 'by' members of the public rather than 'to', 'about' or 'for' them," 13 PPIE aims at emancipating patients to become equivalent and competent members of and contributors to the health care system. By considering and valuing the perspective of those affected, the one unifying goal should be accomplished, namely optimizing care and research by ensuring that the patients' needs are met and their opinions are appreciated. 13 Due to the multi-facetted community and the great variety of stakeholders and research topics, PPIE can take various forms. 14 On the one hand, there are different roles that members of the patient group can embrace, including "individual patients/carers" (person/ family living with a disease), "patient advocates" (representing big(ger) groups of patients), "patient organization representatives," and "patient experts" (individual patients with additional specific expertise). 15 On the other hand, there are different ways and degrees to which the patient group can contribute. While participation (taking part in studies or clinical trials as study subjects) is an indispensable part of research, engagement (the dissemination of information and knowledge about research) and involvement ("active partnership between" 13,16 researchers and patients, patient representatives or the public throughout the research process) are less common. [17][18][19] The choice of appropriate PPIE is especially relevant in pediatric oncology which is a field coined by the necessity of close interdisciplinary collaboration of various HCPs (doctors, nurses, psychologists, social workers, therapists, pedagogues, etc.) as well as a great variety of stakeholders on the patient side. These include patients, survivors, parents, other family members, patient organizations, and patient advocates, among others. Despite the increasing estimation of the benefits and value of engagement and involvement especially in the early stages of research, there is still insufficient implementation with clear regional differences. 5,6,9,13,20 Globally there is a call for establishing PPIE as an integral part of research, although the appropriate and most beneficial form must be chosen on a project basis. 12,14 In addition to international differences and the lack of legal enforcement there are other obstacles on a national and institutional level that prevent the successful establishment of well-practiced PPIE.
These include language and socioeconomic barriers leading to miscommunication, poor recruitment strategies for patient representatives, a lack of financial and temporal resources, and deficient understanding and estimation of other stakeholders' perspectives. 5,7,17,21 Furthermore, there are different logics and rationales for practicing PPIE, leading to a risk for tokenism and the abuse of poorly practiced PPIE to legitimize managerial decisions rather than sincerely including patients' perspectives. 5,16,[22][23][24] To ensure that PPIE is not only established but also practiced well, organizations such as the UK Public Involvement Standards Development Partnership 25 13,[25][26][27][28] Of these standards especially the British National Institute for Health and Care Research (NIHR) Standards for Public Involvement 25 and the guidelines by the former NIHR institute INVOLVE 28 have been serving various researchers to evaluate their own efforts of practicing PPIE, 24,29,30 as well as to create concrete frameworks, models, and quality guidance for successful PPIE. 3,15,29,[31][32][33][34][35][36] Various of these studies on PPIE do not only provide theoretical frameworks but also serve as best practice examples themselves since PPIE is both the subject of interest and part of the methodology, by involving patients and the public throughout the entire research process. 19,29,30,33,[37][38][39] In this context, review papers of studies implementing PPIE highlight the manifold opportunities for PPIE, 15 29,30,37,[41][42][43][44][45] While the effectiveness of and methods for successful PPIE have accordingly become an increasingly popular research topic, the developed methods are predominantly directed toward adult patients, whereas methods for integrating children into the research process are still lacking. 46 Furthermore, the awareness for and attitude toward PPIE among the various stakeholders in health care has only been investigated by few authors 12 and an even lower number of studies has focused on the complex interdisciplinary field on pediatric oncology. 6,12,46 The limited establishment of PPIE suggests that despite the considerable benefits for patient well-being, health care quality and research relevance, the concept of PPIE is still uncommon, even in academic contexts. Therefore, the research aims of the present study were to: 1. evaluate the current understanding of and the different attitudes toward the concept of PPIE among the various stakeholders of pediatric oncology in European countries, to then 2. design and pilot new PPIE practices as well as to 3. integrate the findings into a PPIE training tool realized as a short film.

| Design
An interdisciplinary steering group including psychosocial and medical professionals, as well as patient advocates and patient experts developed a 3-stage project plan including an orientation, an engagement, and an informational phase. To address both patients and HCP equally and to practice PPIE at every level, a mixed-method design was chosen. Figure 1 visualizes  Based upon prior research on the topic 6,13,15,25,29 as well as the expertise and experience of the steering group, an online questionnaire was developed with two separate versions aimed at the patient group (individual patients, survivors, family members, patient advocates, patient experts), and at HCPs (e.g., psychosocial professionals, medical doctors, therapists, nurses) respectively. The questionnaires were aimed at adults which is why assistance by a legal guardian was recommended for underaged participants. Both versions included the same basic information on the concept of PPIE and were structured equivalently, assessing the 1 respondents' demographic information 2 ; relation to the health care sector (e.g., illness, profession, or family) 3 Table 2.
Throughout the entire workshop day, independent research assis-

| Sample and participants
To incorporate and visualize all aspects of PPIE (participation, engagement, and involvement), the description of the study population will follow these categories to better define and establish the used terms.

| Data analysis
All quantitative and qualitative analyses were performed by the scientific board constituted by a clinical psychologist and a patient expert, with the support of project assistants.

| Quantitative analysis
For this exploratory study, descriptive analysis was used for the questionnaire as well as the categorical data collected during the work-

| Knowledge about PPIE terminology
("This is how well I know the following terms in the context of PPIE") As visualized by Figure 3, knowledge was spread across all levels for each term, although patients indicated lower knowledge of each concept than HCPs. With respect to the term "participation" the selfassessed knowledge was significantly higher in HCPs (t(270) = À3.07, p = .002, d = 0.37).
When describing these terms in their own words, the patient group correctly assigned the concepts in 20.36% for patient involvement, 20.96% for participation, and 8.98% for patient engagement.
For the HCP-group, the percentage of correct description was as fol- Fisher's exact test). Figure 6 visualizes the relative number of answers per category.
F I G U R E 3 Subjective knowledge on a continuous scale from Do not know at all (1) to Know it very well (101), significant differences are marked with *.
F I G U R E 4 Agreement to "Where/How can patients be involved?" in patients and HCPs.
3.2 | Stage 2: Engagement "What unmet needs and insights can be assessed using engagement methodologies"

| Open space
The resulting n = 67 associations considering "research" in general were recorded into the categories "research vocabulary" (70%) (e.g., "enables progress and broadens horizon," "generating new knowledge," and "development"), "PPIE topics" (4%) ("recognizing needs," and "better explanationsno specialist terms"), and "valuations" (25%) ("necessity," "curiosity," and "independence"). When asked for research questions the participants would like to investigate, a total of n = 44 topics was named, which due to the great variety and heterogeneity could not be categorized. Some questions were very specific to one research area such as medication, patient well-being ("The impact of information on patient's psychological well-being"), school reintegration or PPIE ("Which adaptions does the health care system need in order for PPIE to be implemented?"). Others were broader and less specific to one research area for example, "How can we connect different worlds?" or "Why do we speak so little about research?" When asked for "associations with PPIE," a total of n = 43 statements was produced whereof 79% were related to PPIE (e.g., "everybody is an expert!"*, "Communication is the key!"*, "practical integration of different perspectives"*, "teamwork makes the dream work," and "interdisciplinary implementation"). 1
In the children's group less written data was generated since the workshop was more interactive with many creative activities such as drawing their own superpowers and associations.

| Stage 3: Patient-oriented report of outcomes
The collaborative production merging the outcomes of stages 1 and 2 resulted in a selection of metaphors and images for one unifying  Children's group

| LIMITATIONS AND FUTURE DIRECTIONS
Following the rationale of implementation sciences, it cannot clearly be distinguished, whether PPIE is the subject or method of the present project which adds considerable complexity to its description.

CONFLICT OF INTEREST STATEMENT
The authors have stated explicitly that there are no conflicts of interest in connection with this article.

DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request.

ETHICS STATEMENT
In consultation with the Ethics Committee of the Medical University of Vienna no ethical approval was needed for the data collection, workshop, and data analysis. A data protection statement was provided by the ethics committee and all participants gave written consent to the anonymous use of the provided data as well as the recording and analysis of written notes during the workshop.