Understanding a science-themed puppet theatre performance for public engagement in Thailand

Fishy Clouds, a 45-minute non-verbal touring puppet theatre Background: show, was created with the objectives of (1) raising awareness of antimicrobial overuse and misuse (the fact that there simply is a problem), (2) raising awareness of the importance of research with children – including those involving antimicrobials, and (3) producing a science-themed performance of entertainment value and high artistic quality. The show used visual storytelling to bring the research and behaviour around antimicrobial resistance (AMR) to life for a broad range of audiences across different ages, locations, levels of education, and language. In order to understand the effectiveness of Fishy Clouds, we used a Methods: realist-informed evaluation approach. A combination of quantitative and qualitative approaches (semi-structured interviews, focus group discussions and field notes) were used for data collection. We received a total of 880 quantitative feedback forms, conducted 22 Results: semi-structured interviews and three focus group discussions. Our data showed that Fishy Clouds was an enjoyable performance to all audience groups and stakeholders and was generally viewed with artistic integrity. However, its effectiveness was primarily in raising existing awareness about medicine use and health more broadly, rather than specific health messaging concerning AMR and research with children. We found that those with limited background on AMR or research with children, such as school children and Karen ethnic migrants exhibited a wide range of interpretations. A science-themed theatre would function better if it is focussed on a single theme, embedded within a programme of activities and conducted at closed venues. Fishy Clouds showed that science theatre events have the Conclusions: potential to support public health programmes and engage local communities in science research.


Introduction
The Mahidol Oxford Tropical Medicine Research Unit (MORU) was established in 1979 as a research collaboration between Mahidol University in Thailand and University of Oxford in the United Kingdom. Among its new programme of public engagement activities [1][2][3][4] , was a collaboration between MORU and B-floor Theatre, an avant-garde Thai theatre company, to produce a non-verbal performance with a combined theme of antimicrobial resistance (AMR) and research with children. This reflects the current national and international efforts to fight resistance to antimicrobials, such as drugs for malaria and tuberculosis, which has significant impacts on all levels of society with the largest impact on vulnerable groups such as children. These themes also reflect some of the research themes of MORU researchers and their collaborators [5][6][7][8][9] . This project was also intended to coincide with the Thailand national AMR week, which was held between the 14 th and 20 th of November 2016. A non-verbal touring performance was chosen for the play to be accessible for children, those who are illiterate, and non-Thai-speaking audiences close to the border with Myanmar. We chose B-floor Theatre because they specialise in physical non-verbal theatre and we had previously successfully collaborated with them on a science theatre project called Survival Games.
The objectives of the performance were to (1) raise awareness of antimicrobial overuse and misuse (the fact that there simply is a problem), (2) raise awareness of the importance of research with children -including those involving antimicrobials (to help children themselves, provided that the research is useful, carefully thought out and ethical), and (3) produce a science-themed performance of entertainment value and high artistic quality.
In May 2016, the B-floor Theatre team started their research phase, which included conducting interviews with MORU scientists and others with interests and expertise in the themes of the production, such as farmers, parents, pharmacists and paediatricians. The B-floor Theatre team also attended meetings with the organisers of the national Thailand Antimicrobial Resistance Week 2016.
The resulting product was a 45-minute, puppet show entitled Fishy Clouds. The play was non-verbal, but it included live music whose story focused on antibiotic overuse (e.g. personal use, agricultural use) by the lead character, Don and the people around him, thereby contributing to antibiotic resistance (Supplementary File 1). An accompanying multi-lingual leaflet was produced to provide audiences with information about the storyline and the main themes of the play (Supplementary File 2). MORU researchers provided oversight of the scientific accuracy, but were not involved in the creative aspects of the play or the storyline.
The performances took place between 11 th November and 14 th December 2016 at nine venues, six in the Bangkok metropolitan area and three in Tak Province close to the Thai-Myanmar border. The venues included schools, theatres, healthcare centres and various open spaces. Some of the performances coincided with events, such as the AMR week launch in Thailand and a university open day. All eleven performances were free. Two performances were held in a theatre in Bangkok and were ticketed with a maximum number of 50 people at each performance. Other venues were open spaces, such as the waiting area of healthcare centres. The latter were not ticketed and numbers were unrestricted. These venues were chosen by both the MORU and B-floor Theatre teams to allow us to be inclusive and to reach a wide range of audience members, including those who do not normally go to see plays as a routine.
In order to understand the effectiveness of Fishy Clouds, we used a realist-informed evaluation approach 10,11 . Rather than seeking a one-off verdict about the play's average effect, as with traditional evaluation methods, realist research recognises that different audiences may react differentially to an intervention. Specifically, realist evaluation seeks to understand "what works for whom in what circumstances" [10][11][12] . This framework provides an explanatory model that links the context, mechanism and outcome configurations, which are hypothesised prior to, or during the implementation of a project 10,11 . Mechanisms "pertain(s) to what turns on in the minds of program participants and stakeholders that make them want to (for example) participate or invest in programs. They may be cognitive or emotional responses, typically in relation to program resources being offered" 13 .
We were inspired by the realist framework because it could assess direct and indirect contribution of our intervention, whether intended or unintended. In addition, this approach offers the possibility to address short term changes, so we would be able to describe these changes after the performance. Another advantage is that this framework may increase our understanding regarding whether, how, and why the performance works or does not work. We would be able to differentiate if the intervention works for one group of audience whereas does not work for another group and why. This paper describes the development of Fishy Clouds and the method and results of its evaluation.

Methods
Approval for the evaluation was obtained from the Oxford Tropical Research Ethics Committee (OxTREC Ref.  and from the Faculty of Tropical Medicine Ethics Committee, Mahidol University (TMEC 16-102).
We identified four types of audience groups: (1) Bangkok theatre goers, (2) school children, (3) scientists and healthcare professionals, and (4) Karen and other ethnic migrants in Tak Province. In addition to audience groups, we identified two stakeholder groups: B-floor Theatre and MORU researchers.
Initial evaluation framework Figure 1 shows the three objectives of Fishy Clouds and their related contexts, mechanism and outcomes of the initial evaluation framework 10,12 .
Awareness raising: We theorised that generating awareness involves (1a) a growing interest in, (1b) a desire to learn more about, and (1c) reflection about one's own ideas and values regarding AMR and research with children. We argued, after consulting existing literature, that the principal mechanism underlying these outcomes is the stimulation of active thoughts through the creative expressions of the performance 14,15 . The context of the intervention involved different venues as well as different educational and cultural background of the audiences, which may influence the audience's ability to understand the messages of and reflect on the performance.
Artistic product: We identified two sets of outcomes for this objective, which are audience group outcomes and stakeholder outcomes. Project stakeholders relevant for this evaluation were MORU researchers and B-Floor Theatre team. Audience outcomes included (2a) the acknowledgment of the performances as "art" and (2b) a sense of enjoyment stemming from it, enacted by validating the performances through active appreciation 16 . Stakeholder outcomes included (2c) the production of locally appropriate content, (2d) potentially sustainable relationships, and (2e) organisational sense of success. As for mechanism, we theorised that these outcomes are realised through the collaborative development of creative forms of expression between artistic and scientific project partners 14,16 .

Data collection
We used a combination of quantitative and qualitative approaches for data collection.
A quantitative feedback form (Supplementary File 3) was adapted from an existing B-floor Theatre feedback form. Questions 1 to 3 of the feedback form relate to art and enjoyment outcomes (outcomes 2a and 2b), whereas questions 5 to 7 and 9 to 11 relate to the awareness raising outcomes (outcomes 1a, 1b and 1c). Wherever possible we distributed the feedback forms to all audience members.
Semi-structured interviews and focus group discussions with audience members were conducted to aid the interpretation of the results of the feedback forms, whereas interviews with B-Floor Theatre and MORU staff were primarily aimed at assessing the collaboration aspect in the process of developing the play. Sampling for qualitative study was purposive to ensure that all audience member groups and stakeholder categories (B-floor Theatre and MORU staff) were included. All relevant documents were translated from English to Thai, Karen, and Burmese to be inclusive for potential participants. Field notes were collected during all performances.
Feedback forms were collected immediately after the shows by members of the study team. In the feedback forms, we asked participants who were willing to be contacted for interviews or focus group discussions to leave the contact details (Supplementary File 3).
Interviews and focus group discussions were conducted up to three weeks after each show by members of the study team.

Data analyses
Data from the feedback forms were entered in the MACRO V4 database, and analysed quantitatively, using descriptive statistical analysis. Audio recordings were transcribed verbatim, translated to English where necessary and analysed using thematic analysis. We had a combination of pre-existing themes based on the initial framework and emerging themes, particularly regarding context-mechanism-outcome configurations. Stata 13 was used for quantitative analysis, and Nvivo 11 for qualitative analysis 17, 18 .
In addition to the quantitative feedback questionnaire, field notes and qualitative interviews, we included two newspaper reviews published by theatre critics in the analysis 19,20 .

Results and discussion
Approximately 1,500 people attended the performances and a total of 1,276 evaluation forms were distributed, of which 880 forms had partial or complete information. Table 1 describes the demographics of participants who completed the forms. Table 2- Table 4 show the results of the feedback forms by venues. We conducted 22 semi-structured interviews (14 audience members, 4 B-floor Theatre staff; 3 MORU staff members; 1 theatre critic) and three focus group discussions (school children, Bangkok theatre goers, and migrant workers in Tak Province). Interviews and focus groups discussions were conducted in Thai, English, and Karen according to the language preferred by the participants. Interviews and focus groups discussions lasted between 20 and 45 minutes each.
Awareness raising -outcomes 1a, 1b and 1c The feedback forms (Q5-Q7, Q9-Q11) drew a positive picture in terms of audience member comprehension (above 70%, Table 3 and Table 4), but the qualitative evidence from the audience members indicated that interpretations can be various.
The overall response to the awareness-raising feedback form items was positive among all audience groups with 86.1% (Q5) reported to have an increased interest in medicine use, 81.9% (Q6) reported a desire to learn more about medicine and 78.6% (Q7) had a self-rated better understanding about medicine use. Research with children also received positive results with Q9 -75.1%, Q10 -76.7%, and Q11 -78.3%, respectively. Audience members at the university open day event, which were majority school children aged 12-17 years old, responded most positively to the questions about awareness of AMR (Q5 -93.9%, Q6 -88.8%, and Q7 -90.8%), while the lowest response was from attendees at the international tropical medicine conference (Q5 -66.7%, Q6 -73.3%, and Q7 -50.0%), who were primarily researchers and doctors. Attendees at the AMR day responded most positively to the questions about awareness of research with children (Q9 -90.6%, Q10 -90.6%, and Q11 -84.4%), while the lowest response was from those at the tropical medicine conference (Q9 -50.0%, Q10 -53.3%, and Q11 -53.3%). The drama increased interest in research with children among all types of audience except for attendees at the tropical medicine conference (Q8 -50.0%, Q9 -50.0%).
Qualitative data illustrated that Fishy Clouds was effective primarily in raising existing awareness about medicine use and health more broadly rather than specific health messaging concerning AMR and research with children.

"I think [I want to learn more about] antibiotics -[…] how many types there are and what types that … in fact the medicines that we
take every day or even our food, they never tell us about the ingredients." (male Bangkok theatre goer, 40 years, high school education, semi-structured interview).
As far as the specific messages are concerned, one of the reviewers wrote, "award winning young playright Pattareeya Puapongsakorn's script for this 45-minute non-verbal performance was easy to understand and straight to the point -and the accompanying leaflet made sure that the audience could share this information with others as well" 19 .
However, a handful of audience members appeared to have misunderstood the messages of the play. For example: "This drama teaches me that if I don't take the medicines, the germs will increase and make us sick, then we would have to take even more medicines and get injections and take 3-4 pills every day. So I'm scared. And from then on, I take all the medicines." (female student, 14 years, focus group discussion with school children).
As the play was non-verbal, this misunderstanding may have arisen from lack of background knowledge or that the information in the leaflet was not read, the latter frequently observed by the team during the performances.

Artistic product -outcomes 2a and 2b
Our theory is that the key mechanism to realising outcomes 2a (validation as art) and 2b (sense of enjoyment) is active appreciation of the show ( Figure 2). We were interested in audience as well as critics' views about the artistic nature of the production. Although the artistic value received mixed reviews, audience members and critics thought the play was entertaining. The play also appeared to have the added value of having a potentially positive influence on promoting art and theatre locally.         Table 2). The first question (Q1) in the feedback forms asked the audience how they felt about the drama, with 92.8% answering that that they "like it". The second question (Q2) asked whether they think it is art, with 79.4% agreeing that it is. The third question (Q3), which asked if the elements of the play relate to the audience's culture, received a lower response of 62.0% overall. The play also had a potential additional positive influence on promoting art and theatre locally. The quote supports our hypothesis that the underlying mechanism was stimulation of active thoughts. Fishy Clouds was not only artistic and entertaining but it also has the potential to provide inspiration for future artistic endeavours. Collaborative production -outcomes 2c, d and e We also considered stakeholder outcomes resulting from the arts-science collaboration. The relevant outcomes in our evaluation framework were (2c) the production of a locally appropriate science-themed drama, (2d) potential for sustained relationships between the arts and the sciences, and (2e) a shared sense of organisational learning and success. This section demonstrates the overall project success with respect to the collaborative production (2d and 2e), but cost-effectiveness (2d) and appropriateness of the play for certain audience groups (2c) were questioned. As far as the process itself is concerned, MORU and B-floor Theatre staff acknowledged that the collaborative production was useful and enjoyable: "During the research process, we worked with the research team and met with different doctors that work in this field. And then we also worked with the playwright, yes, to like, to identify the topic that we're interested [in] from the research process with the doctors." (male actor, B-Floor Theatre, 26 years, semi-structured interview).

"I think I quite like it [the process] in the way that they [B-Floor
Theatre] tried to get a lot of researchers, they ask all stakeholders as much as they can and I point to the outside doctors outside of Thailand, the pharmacies, the activist[s] who work on this area and they try to research everything." (senior male senior researcher, MORU, 38 years, semi-structured interview).
Stakeholders perceived the relationships created and reinforced by this project as potentially sustainable (2d), reinforcing the positive collaborative experience.
The following quotes illustrate the potential for future collaborations: "I hope MORU and Wellcome Trust [are] up for it" (senior B-Floor Theatre team, semi-structured interview); "possible if both sides want to" (senior male senior researcher, MORU, 38 years, semi-structured interview). This suggests that the project was successful in developing potentially sustainable relationships between the MORU and B-floor Theatre. Indeed, MORU are hoping to collaborate with B-Floor Theatre again in 2018.
As for the production of locally appropriate content (2c), most stakeholders viewed that the content was appropriate although a little bit too abstract and the narrative was difficult to follow. The theatre group members emphasised that like any piece of art, it was up to the audience to interpret the art, a notion that was quite challenging to MORU scientists. They also said they there were many logistical and technical challenges related to the non-verbal and touring nature of production.
Theatre staff and scientists understood that venues and audience group differences (e.g. different age groups, cultural and educational backgrounds) could influence the successful delivery of the drama's messages. They also acknowledged that the audience might require a background knowledge of the topic and that it is unrealistic that one-off play will be successful in delivering complicated concepts.
Theatre and MORU stakeholders suggested the play would have benefited from complementary information and a discussion of the artistic and scientific content before and after the show. They also noted that it was more difficult to convey the play's messages in open spaces where not everyone watched the play from beginning to the end.
While almost everyone enjoyed watching the life-size puppets and live music, some MORU stakeholders questioned if they are the most suitable or cost-effective way to convey complex scientific topics such as AMR and research with children. Some researchers viewed that it was ambitious to combine AMR and research with children, both complicated subjects.
Most stakeholders maintain the position that it was unrealistic that a single performance can suit all audience types -children, adults, migrants and theatre enthusiasts. This also fits with the realist concept that the same intervention may have differential outcomes for different target populations, in this case, audience members, depending on the extent to which the "active ingredient" or mechanism is triggered for the context of different audience groups.
Augmented framework Figure 2 depicts the augmented evaluation framework informed by our data. For the awareness raising objective, our data suggested that inter-related elements within the mechanism of active thought processes to raise awareness include the following: recognising the topic area within the creative expression, relating the story to background knowledge and education, and interpreting both the written and non-verbal messages in the performance 16 . These elements of the thought process interact with characteristics of the audience, including their background knowledge of the subjects and their cultural background. In addition we noted that the performance venue, as well as how much the complementary information was utilised influenced the interpretation of the awareness-related outcomes and potentially contribute to misunderstanding of the intended messages.
As for the artistic product, our data suggest that the mechanisms underlying the active appreciation were: audience members' conceptualisation of art and health education, the presence of artistic elements to increase attention and appreciation; and inspiration following exposure to the performance 14,15,21 . Our data showed that apart from appreciation of the performance as art and enjoyment the play, the performance also could play a part in the promotion of the arts.
We considered balancing the interests of the stakeholders and mutual learning as important components of the mechanism of collaborative process aimed at production of locally appropriate content, sustainable relationships, and an organisational sense of success 14,15,22,23 . We see organisational learning not only a process in itself but also a desirable outcome in our science-arts collaboration 23 .

Challenges
The team encountered several challenges in the development of Fishy Clouds, as well as the evaluation. The B-floor Theatre team were granted the artistic freedom to create the performance with scientific guidance from the MORU staff. MORU staff provided oversight in order to ensure scientific accuracy but had no input in the creative process. The collaboration was challenging for scientists who were inclined towards health messaging, such as asking audience members to not take antibiotics for viral fevers instead of merely raising awareness. Another challenge during the development of Fishy Clouds was to strike a balance between the awareness raising objective and the artistic objective.
With the passing of King Bhumibol Adulyadej of Thailand in October 2016, and the resulting nationwide mourning period, the initial performances of Fishy Clouds had to be delayed. This proved challenging to cancel and reschedule these performances and find alternative venues. As a result, our performance venues were chosen primarily based on availability of the performers and venues rather than target audience groups.
The play itself was visually strong, but informal feedback with audience members during the first few performances revealed that there was a feeling that the narrative could have been challenging for some viewers to understand. In response to this, a very basic synopsis (Supplementary File 1) was created in Thai, English, Burmese and Karen and was distributed with the leaflet at the start of each show. The later shows also included a short talk about AMR and research with children by a MORU staff member to orientate the audience.
Another challenge was that we encountered some delays in the ethics approvals, which meant that the qualitative part of the work was only conducted several weeks after the performances resulting in possible recall bias. This also resulted in a temporal mismatch between the feedback forms and the qualitative data.
As with most health research, written consent for participation in the qualitative interviews was required. This requirement resulted in delays in conducting interviews with children as we had to wait for the children to ask their parents to sign the consent forms at home.
Because the audiences included Thai, English, Burmese and Karen speaking people, it was necessary to have four different versions of the feedback forms and interview guides, which was a logistical challenge. Interviews also needed to be translated, both as the interviews were being conducted and afterwards for analysis. This also increased the cost of the evaluation initiative substantially.

Limitations
Our evaluation had several limitations. For our feedback forms, we had to compromise having detailed feedback for brevity and ease of administration.
For venues that were open, it was not possible to determine if an audience member watched the entire performance, so if an audience member did not understand the messages of the play, we did not know if it was because the messages were not clear or that they only watched the play partially.
Our study was focussed narrowly on the specific objectives of the live performance and the evaluation period was limited to the project period. B-floor Theatre created a Youtube video of the show and shared it widely. The Fishy Clouds project was part of a wider programme that coincided with the AMR week in Thailand in 2016. We did not evaluate the impact of the video or the wider or longer term implications of the project, such as the publicity and media coverage that the shows generated.
We also acknowledge that the evaluation team was not completely independent from the team that commissioned Fishy Clouds.

Conclusions
The realist evaluation found that those with limited background on AMR or research with children, such as school children and Karen ethnic migrants on the Thai-Myanmar border, exhibited a wide range of interpretations. A science-themed theatre would function better if it is embedded within a programme of activities, which could include information campaigns. For future projects, we could collaborate with schools or existing healthcare institutions to offer an extended series of activities rather than isolated, one-off events. Future projects should consider narrowing the scientific focus and target audience groups, and taking into consideration the target groups' background knowledge, existing conceptions and exposure to the topic. If possible, the performances should be conducted in closed spaces to increase the likelihood that audiences watch the performances from beginning until the end without interruption and that they participate in related discussions.
As expected, most scientists and healthcare professionals did not report that they had an increase of awareness of the subjects discussed in Fishy Clouds but rather, found the play entertaining and inspiring.
Bangkok theatre goers who had comparatively higher education and went to see the play at a closed theatre appeared more likely to reflect on and understand the messages of the play.
Our data showed that Fishy Clouds was an enjoyable performance to all audience groups and stakeholders and was generally viewed with artistic integrity. However, its effectiveness was primarily in raising existing awareness about medicine use and health more broadly, rather than specific health messaging concerning AMR and research with children. Competing interests PYC is the head of the department that commissioned the Fishy Clouds project and is the Principal Investigator of the evaluation study.

Grant information
Fishy Clouds and its evaluation was funded by a Wellcome Trust Provision for Public Engagement grant (106698) and Oxford's Public Engagement with Research Seed Fund (supported by Oxford's RCUK Catalyst Seed Fund Award).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
groups? How many participants per focus groups and in total across all focus groups? -this is all crucial information which is not included at the moment.

Discussion
The discussion raises interesting and pertinent points, but it needs to go further. It also needs references and support from the relevant literature -there are no references at the moment! P12: an interesting point raised regarding how the abstract nature of the play was a challenging notion to MORU scientists -this really is very interesting although it's not presented in that way. There's no data, quotes, nothing to support this vague statement. This interesting point needs to be properly presented and developed, as well as supported by the relevant literature.

Department of Health Systems and Research Ethics, KEMRI/Wellcome Trust Research Programme (KWTRP), Kilifi, Kenya
The team has written a good report of their intervention. The research methods are well outlined; and the research is transferable. My main comments are as follows: Fishy Clouds is an interesting theatre/musical performance that is attractive to audiences. The development of the storyline included collaborative scripting between B-Floor Theatre team and scientists from MORU, which is commendable.
The main aim of the projects as well as specific objectives are well outlined.
The intervention has been placed within a context that is suitable for the audience; and there has been some attempt towards segmenting the audiences (Children, Adults, Urban populations, Migrant workers) for better effect/impact.
The Realist Evaluation framework used has been well outlined and is appropriate for describing the outcomes of the evaluation study. The study team wanted to learn what works best for which group of audiences; and this is well captured in their findings.
A few areas that the authors could explain further: On which theory was the intervention grounded? For example, were they Theory for the intervention: basing their intervention on the Theory of Change? If they did, it would be good to mention the theory in their paper.
what process of consenting was followed for children to participate as Data Collection from Children: audience members, as well as participating in interviews/focus group discussions? Some conducting a similar study may want to know some of the hurdles the team encountered while involving children.
On page 5 one of the paragraphs makes reference to ' '. It is not clear whether this one of the reviewers means the audience members/respondents?
The evaluation also partly relied on 'critics' to give feedback on the artistic style of Fishy Clouds; who are these critics, and how were they selected?
In their conclusion, the authors have correctly recommended narrowing down both content and audience members; a key strategy for use of arts to convey information, or stimulate dialogue.
Overall, I have enjoyed reading the paper, and think the evaluation has captured very well the findings for each of the 3 objectives.

Is the study design appropriate and is the work technically sound? Yes
Are sufficient details of methods and analysis provided to allow replication by others?