Evaluating the Design of Behavior Change Interventions: A Case Study of Rhino Horn in Vietnam

Behavioral change interventions are increasingly widely used in conservation. Several projects addressing rhino horn consumption were recently launched in Vietnam. We used key informant interviews, document analysis, and marketing theory to explore their strategies for intervention design, implementation, monitoring, and evaluation. We developed a framework to evaluate whether they followed best practice and identify implementation challenges. Interventions could make greater use of key project design steps, including basing interventions on robust research to understand the behavior in question, identifying the target audience whose behavior interventions aim to change, and developing measures that can provide reliable evidence of success or not. Challenges include the need for law enforcement to complement campaigns; improving cooperation between NGOs; and clearly defining aims of demand‐reduction initiatives. Using best practice from other fields and considering demand reduction within the wider context of wildlife, trade policy will help address these challenges.


Introduction
Conservation has suffered from a lack of proper design and evaluation, resulting in an ineffective use of resources and impacts that cannot be measured (Salafsky et al. 2002). Well-designed projects are based on clear objectives, explicit theories of change, supported by research, with built-in monitoring and evaluation, thus enabling evaluation of outcomes, learning, and adaptive management (Stem et al. 2005, Margolouis et al. 2009, Sweeney 2011. However, examples of conservation interventions that fulfill these design criteria are still limited (Ferraro & Pattanayak 2006), where "intervention" means specific actions either as part of an ongoing program or in the form of a short-term project.
In recent years, conservation science has started including the human component more systematically, including understanding constituents underlying human behavior and how to influence them (St. John et al. 2010, Milner-Gulland 2012. One area of particular recent focus for behavioral change interventions is the illegal trade in wildlife (Verissimo et al. 2012, Gao & Clark 2014. Interventions to tackle the illegal wildlife trade have focused mostly on controlling the supply and regulation of these products (i.e., through the Convention on Illegal Trade in Endangered Species of Wild Fauna and Flora [CITES]; Gao & Clark 2014); however, this does not address the root of consumer demand (Drury 2009).
Demand for wildlife products has long existed in Asia. However, markets have expanded dramatically in countries like China and Vietnam, where recent economic prosperity has enabled a larger proportion of the population to afford wildlife products (Nijman 2009). As populations of well-known species such as rhinos and elephants decrease (Emslie et al. 2016), their parts and by-products may be perceived to be more rare; hence, demand and therefore poaching of these species may increase further (Hall et al. 2008, Challender & MacMillan 2014. The rise in the illegal wildlife trade threatens biodiversity as well as the national and international security of the countries involved, which is why addressing the demand for these products is now considered a priority for conservationists and governments (London Declaration 2014).
Vietnam has a long history as an exporter, consumer, and transit country for wildlife products (Nash 1997). Due to its recent economic growth, demand for wildlife products has significantly increased, particularly for rhino horn (TRAFFIC 2013). In response to the recent rapid increase in rhino horn sales, several campaigns have been launched to reduce demand for rhino horn among Vietnamese consumers. This concentration of projects all addressing the same issue in the same country over a similar time-period provides a useful case study for exploring the design and implementation of conservation interventions. It also enables us to discuss with the implementing NGOs their experience in effecting behavioral change among consumers, hence providing useful lessons for future projects.
This study seeks to: (1) evaluate the approach taken by demand reduction campaigns in Vietnam, in terms of the design, implementation, and evaluation of their projects; (2) understand the aims and target audience of demand reduction campaigns, and the extent to which these aims have been met; (3) explore the challenges and opportunities for demand reduction campaigns in Vietnam, and (4) draw general lessons for behavioral change projects more broadly.

Methods
We acquired as much information as possible regarding the planning and design of behavior change interventions for rhino horn in Vietnam, using a combination of interviews in person, literature reviews, and online searches. The list of interventions was based on Sharif (2014) with additions from internet searches and personal communications. Each intervention was linked to the principal implementing organization, and the relevant contact points were approached to participate in the study (Tables 1 and S1). CHANGE is WildAid's implementation partner in Vietnam.
We carried out 10 face-to-face interviews with contact points for each one of the 10 interventions from June 2 to 15, 2015. Interviews were open in format, each referencing a specific intervention aimed at demand reduction for rhino horn. Prior to the interview, the research aim was explained to the respondent, and free, prior informed consent was sought. The interview was recorded with permission, and interviewees were asked whether they were happy to be quoted on behalf of their organization. Those who wished to review the information they had provided were subsequently sent a transcript of their interview for approval. All organizations were happy to proceed on this basis, other than Education for Nature Vietnam (ENV), whose results are therefore not presented here.
The grounded theory approach was used to carry out interviews; this method does not test existing hypotheses but rather uses empirical data to generate concepts and construct theories thereby permitting interpretation and themes within the data to be discovered (Charmaz 2006, Hussein et al. 2014. This approach allows flexibility and relevant follow-up questions to arise in each conversation to obtain as much understanding as possible (Boyce & Neale 2006). The information was then divided into themes and categories to construct a picture of the process each intervention went through (Dworkin 2012).
Next, a project implementation wheel was developed, illustrating the key components required for an intervention to achieve and demonstrate behavioral change ( Figure 1). This was created with elements of project design extracted from the literature of behavioral change, conservation, and business interventions (Margoluis et al. 2009, Zhuang 2013, Osterwalder et al. 2014, later combined with themes emerging from the conversations with respondents. Each intervention was then compared against this wheel to analyze the degree to which projects included these components in their design, based on the interviews as well as material from the organizations' Web sites.

Results
All the interventions included some components of the design framework set out in Figure 1, but only one included all of the components ( Figure 2).

Measurable objectives
Only one intervention articulated measurable objectives (Table S2). Four consumer research studies were carried out, of which only two set out to produce baselines. The objectives set out by the other seven interventions could potentially help determine the success of an intervention, but only these two could compare prevalence of rhino horn consumption pre-and postinterventions in an impact evaluation.

Research and target audience
Although every intervention defined a target audience (Table S3), most did not base this on their own research, instead used results from other organizations' studies. "Target audience" does not necessarily refer to rhino horn consumers, but can include the general public, to create social pressure on consumers or government. Motivations behind rhino horn consumption were generally not thoroughly explored; only four interventions surveyed Vietnamese citizens to understand reasons for consumption.
Three of the four interventions which carried out consumer research prior to the implementation used consultants; TRAFFIC worked with IPSOS, an international research company which operates in Vietnam, while WildAid and the Humane Society International (HSI) worked with Nielsen, an American global information and measurement company which also operates in Vietnam. Breaking the Brand carried out their own research.

Evidence-based messages, behavior change model, and theory of change
Interventions generally lacked strategic planning; only three delivered messages based on evidence that these messages might have the desired effect on the target audience. Organizations made different choices regarding target audiences to reduce demand. While some interventions (like WildAid/CHANGE and Wild Act) focused on the general public, and aimed to raise the profile of rhino horn consumption so as to create public pressure on consumers to change their behavior, others like the Wildlife Conservation Society (WCS) have targeted law enforcers to improve commitment to tackling the issue. Furthermore, assumptions were made regarding how to achieve behavior change. For example, the CITES Management Authority (CITES MA) and HSI have expected to raise awareness among, and potentially change the behavior of, parents of school children who received the "I'm a Little Rhino" book; two million copies of this book were distributed through the country (T. Telecky, pers comm).
Only two interventions were based on theory: TRAF-FIC used a combination of models including Roger's Diffusion of Innovations Model (Rogers 1995), Vlek's Needs Opportunities Abilities Model (VIek 2000) and the National Social Marketing Center's benchmark criteria (NSMC, n.d.). Breaking the Brand used the Spiral Dynamics Model (Beck & Cowan 2014). The remaining interventions relied on past experience to guide their strategy; Chau Nhi from CHANGE commented, "We don't base strategy on theory or model but on experience."

Indicators of success
Although one of the interventions identified indicators of success, these do not follow a theory of change or behavior change model. Several other indicators have been measured and treated as evidence of success (Table 2), but many of these measure progress (outputs) rather than success (outcomes or impact). Social media has been used by 3 out of the 10 organizations to gauge interest, engagement, and activity; for example, the number of posts and articles shared and the number of Vietnamese followers on Facebook. Pledges have been used as an indicator of success, based on commitments to either abstain from buying or using rhino horn or to help spread the message, although none of the interventions could provide evidence that signing pledges leads to behavior change. WildAid has collaborated with well-known Vietnamese artists who publicly signed a pledge committing to reject the use of rhino horn. Workshops led by HSI and CITES MA conclude by asking participants to sign a similar pledge. TRAFFIC also included pledge-signing to engage participants into the Chi community.
Anecdotal information is also widely used, particularly, for informal pretesting of messages; including using reactions observed at workshops or during interviews to identify which messages provoked the desired reaction. Anecdotal evidence includes: r Women in workshops are very grateful to learn that rhino horn does not have medicinal properties and promise to spread the word (T. Telecky, pers comm).
r Upon hearing that rhino horn might be poisoned, one man said that his friend whose son suffers from cancer would finish the piece of rhino horn in use but would not purchase any more after that (L. Johnson, pers comm).
The number of people reached has also been used as an indicator of success, measured via social media, telephone companies, exposure on TV channels, or workshop attendance. This is often considered a way to tell if a message is effective or not, even though it does not indicate whether those exposed changed their behavior or were even consumers to start with. ." "High level commitment to an issue will change people's behavior overnight." "Without stricter laws that it's illegal to consume [rhino horn], demand reduction can't be successful." "There's corruption at every level." Lack of collaboration "People are not connecting, talking or sharing to ensure there is no overlap or to be as effective as possible. Why do different organizations give different messages?" "It's a turf war." "[There is] redundancy and overlapping of what's being said." Clarity of purpose "Our campaign has gotten criticism that it's just awareness raising and not demand reduction but awareness raising is the methodology, objective is to change behavior. Can't differentiate because you need awareness raising and policy to achieve demand reduction." "By changing people's awareness they are automatic to change their perception." (On awareness raising) "That's what you need: you repeat, you repeat, you repeat, you reinforce. The more people you hear it from, the better." "'Demand' is the preference, not the actual consumption." "Demand reduction is like a fashion, everyone is doing it, but what is it and what do you aim for?" Practitioners' quotes on the three different challenges identified to achieve demand reduction for rhino horn in Vietnam.

Evaluation
The two interventions that created baselines to evaluate prevalence of rhino horn consumption; pre-and postintervention are the only ones so far that can discuss results. Although TRAFFIC's Chi campaign has not been evaluated yet, as it is still in its implementation phase, HSI published the results of their joint campaign with the CITES MA in 2014, suggesting a decrease in rhino horn demand. However, there were doubts about the degree to which inferences can be drawn given that the research was not adequate to attribute impact in the presence of other high-profile campaigns, and based on small sample sizes without a power analysis or without consideration of the likelihood of biased responses due to the sensitivity of the behavior (Roberton 2014).
Some practitioners were doubtful whether evaluation measures of behavior change interventions are feasible: "Results can't be 100% correct, understandably. Perception [is] very abstract." -Thuong Nguyen, CITES MA.
"Few methods available to evaluate impacts of a campaign effectively. In addition, impacts cannot be immediately measured because these campaigns don't show impact or results immediately." -Van Nguyen Dao Ngoc, WWF-Vietnam.

Challenges of implementing behavior change
The three most pressing difficulties that were discussed by the practitioners were: lack of law enforcement, cooperation, and clarity of purpose (Table 3). Lack of enforcement was brought up by seven out of the nine organizations; without the appropriate intervention from law enforcement agencies, reducing the demand for illegal wildlife products is not possible. Serious commitment is needed from the government to address the issue of trade and consumption of wildlife products in Vietnam and elsewhere.
Half of the practitioners mentioned the lack of cooperation between organizations working on this issue in Vietnam. While some talked about interventions overlapping and competing for territory, others highlighted that there could be benefits to targeting different groups with different messages, since there is no one approach to changing everyone's behavior. On the other hand, some practitioners did not even think it is possible for organizations to work together to deliver one single message because of lack of coordination. The clarity of purpose, such as distinguishing between "demand reduction" and "awareness raising," was also described as a problem. There were different ideas of what should be the aim of demand reduction work (awareness raising vs. behavior change) and there were even different opinions on how demand reduction should be defined (Table 3).

Discussion
A key limitation that this assessment has highlighted is the lack of behavior change project design elements in the demand reduction interventions under study. Only one intervention had defined measurable objectives, which is a crucial component in project design (Sexton Marketing Group 2007). Having clear, time-bound objectives helps define the activities and tools that will be needed to carry out the project successfully (Salafsky et al. 2002) as well as giving something to evaluate against at the end of the campaign (Powers 2004, Sweeney 2011. Defining a clear target audience is of the upmost importance for interventions which aim to change a specific behavior (Michie et al. 2011, Osterwalder et al. 2014. Marketing principles are useful here; in a marketing campaign, the target audience is at the centre of the research since every activity must be guided by this demographic's perceptions, attitudes, drivers, and behavior (Smith & Strand 2009). If an intervention aims to influence a behavior, the behavior in question and the audience performing it must be understood as much as possible (St. John et al. 2010, McKenzie-Mohr 2011. This requires preintervention research, which can also establish baselines against which to compare postintervention data for evaluation (Ferraro & Pattanayak 2006).
The different target audiences being addressed in Vietnam suggest that there is still no consensus as to who the main consumers are, and their motivations for consumption. Some interviewees felt that traditional medicine was the most pressing use to address, while others focused on the luxury market. It may be that this segmentation is appropriate to ensure that both types of use are covered by different campaigns, but little evidence was found to suggest that a strategic approach was being taken between NGOs. Several of the interventions have a general public audience, in order to raise awareness rather than change behavior of specific consumer groups. In some cases, awareness raising is seen to be a way to change behavior, but behaviors are not just influenced by awareness about a subject but also by the social context, political and cultural forces, and even the degree to which the source of information is trusted (Drury 2009). As Verissimo et al. (2012) point out, the links between delivering information and behavior change "are tenuous at best." Basing an intervention on a behavior change model avoids guesswork when developing the messages to be delivered (Cramphorn 2004, Smith & Strand 2009). As Ferraro and Pattanayak (2006) state, "you cannot overcome poor quality with greater quantity;" which is why creating the right messages to target the relevant audience is essential and why these must be pretested before delivery (Powers 2004, Duthie 2014. Additionally, if message contain new information that does not feel right, the target audience might unconsciously disregard it or adapt it to fit previous beliefs (Cramphorn 2004), which can be severely detrimental in the case of wildlife consumption. For example, if the prior perception is that animal-based medicines enhance and maintain one's health and communication materials are inadequately designed, users can adapt these messages to support these beliefs.
The lack of behavior change models and strategic planning in our case study interventions has led to a confusion between indicators of success and evidence of progress. Margoluis et al. (2009) andSweeney (2011) emphasize the importance of ensuring that there is an adequate theory of change. For example, although there is evidence that pledges are effective ways to encourage behavior change (Mckenzie-Mohr 2011), the current measures of success (Table 2) do not get at actual behavior change. Building evaluation into project design from the start provides a framework from which knowledge can develop to inform and improve future projects. However, urgency tends to lead to evaluation coming in as an afterthought so that the necessary information has not been collected to attribute impact (Margoluis et al. 2009).

Recommendations
Valuable lessons in behavior change and evaluation methods can be drawn from other fields. The public health literature highlights the importance of research being consumer-oriented to enable greater understanding of the target audience and the behavior in question (Jones et al. 2005). Marketing research uses both qualitative and quantitative methods to design and evaluate projects (David Ogilvy Awards 2011). Within development, institutions like the World Bank have substantial expertise in implementing robust approaches to program design that allow impact to be evaluated, such as randomized controlled trials or quasi-experimental methods (Haynes et al. 2012). A behavior change intervention for wildlife products would be an ideal candidate for these types of robust design, which could then form the basis for learning about what works for the sector as a whole.
As highlighted by interviewees, behavior change is part of a wider intervention landscape which also includes law enforcement and legislative change. Collaboration among the different entities working to tackle this issue is incredibly valuable; mistakes will be made, but if these are shared within an implementer community, learning and adaptation can take place. Lessons learned from other fields, which demonstrate that there are no substitutes for baselines, robust research and behavior change models, can be used to improve conservation interventions. Although demand reduction for endangered species in Vietnam faces several challenges, there are already existing initiatives and resources which could help to mitigate these (summarized in Table 4). Understanding how to effectively change the behavior of consumers of illegal wildlife products and learning from previous interventions is critical to successfully reduce demand for endangered species.  (2014), which brought world leaders together to make commitments to fight trade in endangered species (https://www.gov.uk/government/topical-events/illegal-wildlife-trade-2014) Lack of collaboration between NGOs r Grants requiring partnerships and collaboration between organizations will result in organizations seeking each other out to work together. These grants should include budget to hire a third-party (individual or agency) expert in collaboration/partnerships between organizations (Glennie 2012).
r Sharing of resources and information. For example, TRAFFIC's initiative online tool-kit to share information, lessons learned, and provide guidance for this kind of work (http://www.changewildlifeconsumers.org/). r Organizations can refer to tool-kits for partnerships and collaborations such as the one provided by Kara (2014).
Lack of clarity of purpose r Grants requiring partnerships and collaboration between organizations will result in organizations seeking each other out to work together. These grants should include budget to hire a third-party (individual or agency) expert in collaboration/partnerships between organizations (Glennie 2012).
r Sharing of resources and information. For example, TRAFFIC's initiative online tool-kit to share information, lessons learned, and provide guidance for this kind of work (http://www.changewildlifeconsumers.org/). r Organizations can refer to tool-kits for partnerships and collaborations such as the one provided by Kara (2014).
Suggestions as to how organizations can overcome the three main challenges identified in the research study.

Acknowledgments
We would like to thank the Department for Environment, Food & Rural Affairs (DEFRA) project WC1101, for their support of this research study; Gayle Burgess for her help and guidance; and all of the practitioners we interviewed for this project.

Supporting Information
Additional Supporting Information may be found in the online version of this article at the publisher's web site: Table S1. Contact details of practitioners responsible for each one of the interventions analyzed. Table S2. Summary of different objectives set by the organizations. Table S3. Summary of different target audiences.