Material practices for meaningful engagement: An analysis of participatory learning and action research techniques for data generation and analysis in a health research partnership

Abstract Background The material practices which researchers use in research partnerships may enable or constrain the nature of engagement with stakeholder groups. Participatory learning and action (PLA) research approaches show promise, but there has been no detailed analysis of stakeholders’ and researchers’ experiences of PLA techniques for data generation and co‐analysis. Objectives To explore stakeholders’ and researchers’ experiences of PLA techniques for data generation and co‐analysis. Design The EU RESTORE implementation science project employed a participatory approach to investigate and support the implementation of guidelines and training initiatives (GTIs) to enhance communication in cross‐cultural primary care consultations. We developed a purposeful sample of 78 stakeholders (migrants, general practice staff, community interpreters, service providers, service planners) from primary care settings in Austria, England, Greece, Ireland and The Netherlands. We used speed evaluations and participatory evaluations to explore their experiences of two PLA techniques—Commentary Charts and Direct Ranking—which were intended to generate data for co‐analysis by stakeholders about the GTIs under analysis. We evaluated 16 RESTORE researchers’ experiences using interviews. We conducted thematic and content analysis of all evaluation data. Results PLA Commentary Charts and Direct Ranking techniques, with their visual, verbal and tangible nature and inherent analytical capabilities, were found to be powerful tools for involving stakeholders in a collaborative analysis of GTIs. Stakeholders had few negative experiences and numerous multifaceted positive experiences of meaningful engagement, which resonated with researchers’ accounts. Conclusion PLA techniques and approaches are valuable as material practices in health research partnerships.


| INTRODUCTION
Involving patients and communities in health research partnerships is consistent with international policies and is recommended for ethical and instrumental reasons. 1,2 It is increasingly a requirement for research funding in many countries. There is a long-standing awareness that meaningful involvement is a genuine challenge. 3,4 Recent reviews show persisting concerns that current practice is tokenistic. 5 Our working definition of "meaningful engagement" is an experience of partnership in research that is collegial, inclusive and active for participants. Meaningful engagement reduces asymmetries of power, encourages participants' ownership of the project and enables participants' authentic perspectives to emerge clearly in research outcomes. [6][7][8] "Material practices," such as the types of methods and techniques that researchers use to involve stakeholders, can enable or constrain participation in research. 9 Therefore, while there are valid concerns about a "flight to empiricism" and an overemphasis on "how to" manuals, 4,10 it is important to determine what methods and techniques are used to frame the interactional and relational nature of partnerships.
This will allow identification of material practices which minimize tokenism and enhance opportunities for meaningful engagement.
Research to identify the best methods to achieve meaningful engagement is currently lacking. 11 Boote et al. 12 reported that group meetings were the most common method used to engage the public.
Workshops, meetings and focus groups were identified as common methods of engagement in three other reviews. [13][14][15] Domecq et al. 11 found that the most common methods in use were focus groups, interviews and surveys.
Tierney et al.'s 5 review of service user involvement in academic primary care also found that interviews and focus groups were commonly employed. It reported examples of studies which had used methods from the field of participatory health research and found that the use of participatory methods was more congruent with stated aspirations for meaningful engagement than "standard" research methods were.
Participatory health research is an overarching term that refers to "bottom-up" research approaches specifically designed for stakeholder involvement in research partnerships. These include, among others, participatory research (PR), [16][17][18] participatory action research (PAR), 19,20 community-based participatory research, 21,22 participatory rural appraisal (PRA) 8,23,24 and participatory learning and action (PLA). 25,26 All share a democratic ethos, are strongly committed to meaningful engagement by stakeholders and promote research partnerships that strengthen relations between academy and community.
Participatory approaches emphasize the need for stakeholders' active engagement across the full range of research activities, including data generation and data analysis.
Participatory approaches face challenges, such as the need to see community participation as a long-term process of implementation and support for improved health outcomes 27 and the fact that many professional health researchers may be unprepared for the reversals of power and hierarchical relationships that a participatory approach may require. 28 Notwithstanding these challenges, there is consistent evidence that participatory approaches provide added value in terms of shaping the purpose and scope of research, improving research implementation and enhancing both the interpretation and the application of the research outcomes. 29 Furthermore, participatory approaches offer a range of interesting and interactive material practices and techniques. PLA is noteworthy in this regard. This is a form of action research rooted in the interpretive and emancipatory paradigms. 25,26 Based on the work of Robert Chambers, PLA is a methodology which offers a practical approach to research where asymmetries of power may exist. 7,8,[23][24][25]30,31 It involves a combination of a PLA mode of engagement and PLA techniques. A PLA mode of engagement aims to create a trusting relational environment, a "safe space" where stakeholders are encouraged to respect a diversity of views and experiences, and to learn from each other's perspectives. 30,31 All stakeholders are considered to possess expert knowledge about their own lives and conditions which they bring to the "stakeholder table" for a PLA brokered dialogue, where, using various PLA techniques, implicit knowledge becomes explicit and much that otherwise might remain hidden emerges.
PLA techniques evolved originally from PRA and are based on a shared stock of ideas and experiences from participatory trainers and stakeholders around the globe. They continue to be adapted to specific contexts as required. 31 The techniques are recognizable as PLA techniques because they are explicitly designed to be active, inclusive, user-friendly and democratic. They are visual and tangible, meaning that they are used to generate physical maps, charts and diagrams (described further under Methods). Generation and co-analysis of data go hand in hand and are best understood as a structured, integrated process. Stakeholders' priorities and perspectives are meant to guide the generation and co-analysis of data about the issue being explored, with researchers acting as catalysts rather than directors or top-down decision-makers. had few negative experiences and numerous multifaceted positive experiences of meaningful engagement, which resonated with researchers' accounts. There are some recent positive examples of PLA applied to primary care health research. These studies describe meaningful involvement of migrants and other stakeholders in the development of a guideline to improve communication in cross-cultural consultations; 32,33 involvement of people with aphasia, speech and language therapy educators and students in the evaluation of community services for people with aphasia; 34,35 and involvement of a variety of marginalized groups (sex workers, homeless people, Irish Travellers, migrants and drug users) in the identification of priorities for primary care team activities. 36 However, there has been no detailed analysis of stakeholders' or researchers' experiences of PLA techniques for data generation and co-analysis used within a PLA-brokered dialogue. Such an analysis would provide important empirical data about the ways in which PLA techniques are experienced as material practices 3,9 and how they shape interactional and relational aspects of health research partnerships.
In this paper, we describe the use of two PLA techniques (Commentary Charts and Direct Ranking) used for data generation and co-analysis, and the perceived utility of these by various stakeholders and researchers involved in a recent European primary health-care implementation project.

RESTORE (Research into Implementation Strategies to Support
Patients of Different Origins and Language Background in a Variety of European Primary Care Settings) was an EU-funded primary health-care research project that ran from 2011 to 2015.
The objective of RESTORE was to investigate and support the implementation of guidelines and training initiatives (GTIs) intended to enhance communication in cross-cultural primary care consultations. This qualitative, comparative case study 37 involved diverse stakeholders across five primary care settings: Austria, England, Greece, Ireland and The Netherlands (see File S1 for a description of the five settings). A sixth research team in Scotland focused on policy-related implications of the study. The choice of these six countries matched the academic teams who developed the proposal and intentionally included countries with diverse primary health-care systems. Ethical approval was granted by respective national committees.
A detailed description of the study protocol is available elsewhere. 38 For the purpose of this study, we emphasize that RESTORE comprised three stages of fieldwork. 3. Stage 3. Stakeholders successfully adapted their selected GTI at a local level and worked on its implementation, with evidence of some impact on daily practice. 41 RESTORE was the overall setting in which we explored and evaluated stakeholders' and researchers' experiences of two PLA techniques.
These techniques were employed during Stage 2 and were intended to enable stakeholders (migrants, general practice staff, community interpreters, service providers, service planners and others) to work collaboratively and with RESTORE researchers to select a GTI for their local primary care setting.
First, we describe sampling and recruitment for RESTORE-this is the sample for the evaluation reported in this paper. We then describe the two specific PLA techniques employed in RESTORE and the methods used to evaluate stakeholders' and researchers' experiences of these techniques. Finally, we present our analysis of the evaluation data.

| Sampling and recruitment
In Stage 2, we used a combination of purposeful and network sampling to identify and recruit 78 stakeholder representatives across five research sites. A geographically defined area (district) was selected in each partner country. Selection was pragmatic, based on researchers' knowledge of groups working in the district and proximity to the research teams, to facilitate data collection. Eligible organizations/ agencies were those involved in primary health-care planning and delivery (eg health-care centres, regional health authorities) as well as those addressing migrant health issues (eg non-governmental organizations focused on migrants). The aim was to identify individuals who were decision-makers (eg health authority service planners and policymakers), service providers (eg general practitioners (GPs), primary care staff, community interpreters) or service users (ie migrants using local primary care services).
In line with standard ethical procedures, stakeholders in all countries were provided with information leaflets and signed consent forms prior to participating in fieldwork sessions.

| PLA data generation and co-analysis in RESTORE
Two members of the RESTORE consortium in the Irish team (practitioner/trainers with over 25 years' international experience in PLA research and training in diverse cultural and social settings) 26,32,33,[42][43][44] led the design of PLA in RESTORE. They provided training and standardized fieldwork protocols which enabled researchers to facilitate PLA in a consistent and rigorous manner across research sites.
The PLA process for RESTORE was based on a PLA mode of engagement and PLA techniques in a PLA-brokered dialogue between stakeholders. One of the striking features of PLA in general is the highly visual nature of the techniques used. Stakeholders work together to generate maps, charts and diagrams which function as powerful reference points (data displays) as they engage in verbal interaction, discussing, questioning, and learning from each other's perspectives, adding new data to maps and charts. 8,32,[44][45][46][47] The inherent analytical capabilities of PLA techniques aim to enable stakeholders to assess, correlate, categorize and/or prioritize data they are co-generating. PLA techniques, therefore, have the capacity to facilitate meaningful engagement that automatically incorporates co-generation and co-analysis of data "by" and "with" stakeholders.
In a very practical way, then, stakeholders using PLA techniques engage in a structured, integrated, visual-verbal-tangible process of co-generating and co-analysing data which produces visual-tangible results. 6,25,32,43 This activity can appeal to a wide range of stakeholder groups, including those where literacy and/or numeracy challenges may feature, as a key aim is to ensure that stakeholders/stakeholder groups do not become disenfranchised during the research process. • Stakeholders record key data on Post-It notes about the issue of interest. This provides a visual representation of their co-generated data.
• The Post-Its are assigned to the relevant category on the chart. These categories may be determined before or during the sessions. This is the start of the co-analysis process.
• Researchers and stakeholders consider and discuss the emerging and completed Commentary Chart. This process automatically incorporates co-generation and co-analysis of data "by" and "with" stakeholders.
• Researchers and stakeholders continue the co-analysis with a visual-verbal-tangible process of "interviewing" the chart. The emphasis is on looking at the Commentary Chart and encouraging stakeholders to share their unique knowledge and insights, to exchange differential knowledge by asking: Does the Commentary Chart make sense? Are stakeholders comfortable with their data display? Is there anything striking/ odd about the data display? Are diverse views sufficiently and accurately represented? Does anything need to be added as we reflect on the Chart?
Are stakeholders willing and content to "sign off" on the Chart? Can it now be presented to another stakeholder group (as needs be) for discussion and development?
Direct Ranking • Physical objects and/or images are selected to represent the issues/entities being ranked. This provides a visual focus for the co-analysis process.
• Stakeholders place the selected images randomly on a large flipchart sheet, to give each image equal visual "weight" and importance.
• Stakeholders engage in co-analysis and clarify what the ranking criterion will be.
• Stakeholders discuss each object/image in relation to the agreed ranking criterion, listening, learning, questioning, reflecting and assessing, thus continuing the integrated processes of data generation and analysis.
• When discussion is complete, each stakeholder is provided with an equal number of "votes" (eg paper clips, coins, matches).
• Stakeholders distribute their votes across the images.
• Stakeholders count the number of votes assigned to each image.
• Results are double-checked, recorded in numerical form on "Post-It" notes and attached to the relevant images. (see Figure 1). For example, in Ireland, stakeholders met in one large group for each of five PLA sessions. They had identified a "local set" of five GTIs and co-generated five separate Commentary Charts.
Where several stakeholder groups met separately (eg Austria) or were geographically dispersed (eg The Netherlands), charts were computerized and circulated around stakeholder groups by email, iteratively accruing additional data. On occasion, researchers took physical charts from one stakeholder group to the next, and data were added incrementally. As Commentary Charts "travelled" around stakeholder groups, the knowledge-exchange and knowledgeenhancing process continued.
The intended practical outcome expected of Commentary Charts in RESTORE was that they would present a visual, tangible data display of stakeholders' knowledge, expertise and perceptions about the sets of GTIs. Stakeholders could then review the data display and continue their co-analysis activity as they began to use Direct Ranking to select a single GTI for implementation at local level.
Direct Ranking is an interactive technique for identifying priorities or preferences in a democratic manner. It yields a visual result in chart form. Box 1 provides a summary of the generic steps for this technique. In RESTORE, the specific application of Direct Ranking was to produce a clear, documented democratic result-a single GTI for implementation that stakeholders are willing to "sign up to" for Stage 3. The images selected to represent each GTI were photographs of the front covers of GTIs. The agreed ranking criterion was "Prioritize the GTIs in terms of the most-to-least suitable for implementation in our general practice setting and context." Stakeholders had equal voting power as they had 20 paper clips each (see Figure 2). The intended practical outcome expected of Direct Ranking Charts in RESTORE was that they would present a visual, tangible data display of stakeholders' decision about which GTI was considered most suitable for implementation in their setting.
Across research sites, the majority of PLA sessions involving Commentary Charts and Direct Ranking were each 2-3 hours in duration. The completed paper-based charts were computerized to preserve them and to make them readily available for further analysis.

| Evaluation of the use of PLA techniques in RESTORE
Stakeholders at all sites participated in qualitative "speed" evalua-  for this topic guide). Interviews were audio-taped and professionally transcribed.
All evaluation data were collated and analysed following the principles of thematic analysis in qualitative research. 48,49 The PLA researchers at the Irish site, who have more than 25 years' experience of qualitative interviewing, generated a "start list" of codes 50,51 derived from participatory research literature describing meaningful engagement (eg active inclusion, collaboration/collegiality, power-sharing) and its opposite (eg exclusion/passivity, researcher-controlled, powerlessness). [6][7][8] This, augmented by repeated readings of researchers' and stakeholders' data, generated a final set of 33 codes. Each code was understood to incorporate its mirror or binary opposite. Data were collated under these codes to identify emerging themes.
The researchers also conducted a basic content analysis to establish the relative weighting of "positive" to "negative" evaluation comments in the final set of themes. 52,53 In keeping with our comparative case study design, the analysis of all evaluation data explored shared and differential findings across the five contexts.
Using different enquiry techniques (speed and participatory evaluations, focus group discussions and team "reflection" interviews) to explore researchers' and stakeholders' perspectives about their experiences of the same events (PLA sessions), we achieved a measure of triangulation, or cross-validation. As per Lincoln and Guba, 49 prolonged involvement of researchers in fieldwork for "trust-building" and "knowing the culture," coupled with persistent observation of stakeholders' reactions to PLA methods and peer debriefing by research teams when producing regular field reports, contributed to study depth. This, in Country of origin conjunction with the triangulation or cross-validation mentioned above, contributed to and enhanced the trustworthiness, credibility and dependability of the study. 49,53

| Study sample
There was appropriate representation of stakeholders at each site by gender, age group, country of origin and type of stakeholder group (see Table 1), thus validating the purposefulness of the sample.
We monitored attendance at each session and over time. There was strong representation of the academic research team, as 16 of 18 individuals participated in PLA fieldwork and its evaluation (see Table 2).

| Emergent themes
We identified five interrelated themes of stakeholders' experiences that elucidate their positive perspective on the PLA techniques (see Table 3). Findings were relevant across countries and participant groups unless otherwise specified.
Stakeholders described their overall involvement in the PLA process of co-generation and co-analysis in ways that speak powerfully of (1) meaningful engagement in a "safe space": active inclusion, collegiality, collaboration. They reported that group dynamics were positive.
The working environment was considered safe, allowing stakeholders to readily and safely express diverse views in a relaxed and enjoyable manner. This was also noted and commented on by researchers.  Researchers noted the equalizing power of PLA, "levelling the playing field," and that stakeholders with a lower level of language skills were not disenfranchised-they could "see" the result; the presence

| DISCUSSION
In this paper, we have explored the perceived utility of two PLA techniques-Commentary Charts and Direct Ranking-for data generation and analysis with a diverse sample comprising migrants, general practice staff, community interpreters, service providers, service planners and academic researchers.

| Contribution to existing literature
There is limited knowledge about suitable methods for involving stakeholders in a meaningful (rather than tokenistic) way in health

Sustained engagement
And also I think they found it fun, the stakeholders around the We also think the [PLA] system will work … because people like these methods, they will go further on with this. NL Researcher #1 R Int T A B L E 3 (continues) research partnerships. 3,11 There is evidence that participatory learning and action research approaches and methods seem promising. stakeholders were predominantly positive about their experiences. In particular, themes 3 and 4 (Democracy-in-action; Power, ownership) reveal the ways in which power imbalances were reduced and that experiential learning, rather than reiteration of professional concepts, became the common ground upon which democratic decision making took place. This is an important function of PLA: to "level the playing field" where asymmetric power relations between stakeholders/ stakeholder groups may exist. 7 Our results show that this levelling also occurred between stakeholders and researchers: throughout the Commentary Charts and Direct Ranking, it was stakeholders who exercised power and took on the key responsibility of selecting a GTI for implementation, thereby "setting the agenda" for the final stage of RESTORE fieldwork.
Researchers, in their capacity as catalysts, facilitated but did not control this process. This shared ownership and agenda-setting takes us firmly beyond tokenism and towards a "shared power" approach, enhancing the research partnership. Therefore, this analysis of two PLA techniques to support such dialogues is an important contribution to the literature.

| Methodological critique and suggestions for future research
We were unable, for site-specific ethical reasons, to include the use of stakeholder evaluation comments from the English site in our thematic and content analyses. However, we were able to ameliorate this by including comments from researchers' reflection interviews.
We cannot claim representativeness of findings for the qualitative study data presented here. However, we emphasize that in this comparative case study, spanning five European countries with very different primary care systems, the same PLA techniques were used and successfully involved diverse stakeholders in data generation and co-analysis.
Regarding the PLA techniques employed, there was variation in our added to the quality and depth of evaluation data. Therefore, we suggest using an array of PLA evaluation techniques in future projects to explore all stakeholders' experiences of involvement in greater depth.
RESTORE ended before our thematic analysis of stakeholders' evaluation data took place, and we did not, therefore, benefit from their contribution to the development of codes and categories for thematic analysis, nor from their potential insights about the relevance and veracity of evaluation results. While we are confident that data saturation was achieved as the analysis reached a point where the codes and themes were comprehensive, we acknowledge the lack of member checking. In future projects, it would be apposite to invite stakeholders to co-generate evaluation criteria and to co-analyse the results of evaluation data, thus closing the circle of "involvement." To add to the evidence base, we need further research and evaluation to explore whether and how PLA techniques might work when applied in projects with very different research foci and stakeholder groups to those in RESTORE. The specifics of involving community and health sector partners in analysis of other qualitative methods, such as interviews and focus groups, would also be valuable.

| CONCLUSION
PLA Commentary Charts and Direct Ranking techniques, with their distinctive visual, verbal and tangible nature and inherently co-analytical capabilities, are rated very positively by stakeholders and researchers. The positive benefits gained from the PLA process in this study (knowledge sharing, knowledge enhancement, levelling the playing field, new knowledge impacting on collaborative decision making) outweighed the negatives. The significant additional investment of resources was impactful and was worth the time and effort.
Therefore, we recommend the use of these two PLA techniques as material practices to enable collaborative decision making and meaningful engagement in health research partnerships.