Organising stakeholder workshops in research and innovation – between theory and practice

This article addresses the theory and practice of creating responsiveness among actors through deliberative dialogue processes with stakeholders from diverse institutional settings. The EU’s decision to mainstream stakeholder deliberation in research and innovation, as part of its focus on responsible research and innovation (RRI), creates a new potential for experimentation and integration of deliberative processes. The article presents a list of essential considerations for three steps in the workshop process: planning and design, workshop interaction and the gathering of conclusions. Finally, the article illustrates the challenges of applying theory to five European stakeholder workshops coorganised by the authors. The illustration highlights the difficult interaction between theory and practice. The article concludes that while theoretical perspectives can provide general guidance, practical experience is essential when dealing with the trade-offs that are an intrinsic part of organising stakeholder workshops.


Deliberation to Increase Responsiveness Among Actors of Research and Innovation
Developments in research and innovation (R&I) are often created through collaboration between diverse actors, and the innovations created can affect actors far beyond the innovation process itself. Through deliberation, all affected actors can be brought together. Deliberation among actors of research and innovation is not something new, yet the current policy initiatives promoting deliberation in EUled research has an interesting potential to mainstream deliberation in larger R&I projects in Europe. There has been a push within the EU to increase dialogue among all actors of research and innovation, including public administration, businesses, and civil society organizations. The dialogue is promoted as part of the objective to create responsible research and innovation (RRI 1 ) in Europe. The promotion of RRI creates new opportunities for deliberation and at the same time defines a context and aim for such deliberation. One key aim is to achieve responsiveness among actors of research and innovation.
The article will examine how existing theory and practical experience with stakeholder workshops can inspire dialogue processes working toward the aim of responsiveness. Thus, both the theoretical literature based on practitioner experiences and the literature on democratic ideals will be applied to explore the complexity of deliberation processes in R&I. To highlight gaps between theoretical work and dialogue practices, the article will use a case, which illustrates the challenges of organizing stakeholder workshops. Through the illustration, the article aims to move discussions of stakeholder workshops to the muddy center, where difficult decisions and practical trade-offs have to be considered. Central to the success of dialogue initiatives is the ability to create a constructive dialogue among the participating stakeholders. The article discusses how something constructive can come out of something as imperfect as dialogue between strangers.
First, a short introduction to the aim of creating responsiveness, the idea of RRI, and the context it creates for dialogue processes within research and innovation is needed. This short introduction will discuss how the focus on RRI changes the possibilities and aims for carrying out deliberative processes in R&I.

The Development of RRI and its Relevance as a Context for Deliberation
1 Some scholars writing in the same context use the term responsible innovation (RI), although neither term has one standard definition RRI has been discussed in academic circles focusing on new developments of research and innovation in Europe and the United States over the last ten years, yet no commonly accepted definition of RRI has emerged. What different definitions have in common is the aim to bring together stakeholders in debating considerations over the risk and benefits of new technologies (Callon & Lacoste, 2011). As an interesting new development, RRI is being promoted as a cross-cutting issue in the EU horizon 2020 research program (R. Owen, Macnaghten, & Stilgoe, 2012). These developments at a policy level happen in parallel to a new and developing literature on RRI with high expectations of what structured spaces for dialogue and collaboration can achieve at a policy level. Participation, inclusion, engagement, deliberation, involvement and dialogue are just some of the terms used to examine interaction between stakeholders in the current debate on RRI (Stilgoe, Owen, & Macnaghten, 2013;Sykes & Macnaghten, 2013;von Schomberg, 2013;Valdivia & Guston, 2015). The central purpose of RRI, according to von Schomberg (2011), is to increase responsiveness among actors, thereby making R&I systems more adaptable to changing circumstances. Central to the aim of responsiveness are shared learning and adaptation processes among the involved actors. This understanding of responsiveness closely resembles the idea of collaborative rationality presented by Innes and Booher (2010, p. 9), which emphasizes "… collective learning that will help make the community more adaptive and resilient." In participatory processes, responsiveness can be seen both as a precondition and a final outcome. As a precondition, responsiveness is a mindset emphasizing a willingness to listen and collaborate. However, responsiveness can equally be seen as a final outcome, where an understanding and appreciation of actor positions and the possibilities for collaboration has been developed (Nielsen, 2016, p. 6). Responsiveness, therefore, emphasizes specific aspects of participatory processes, namely those related to the social impacts.
The ideas being developed under the concept of RRI are not new, but build on many years of practical experience with technology assessment in a European context as well as inspiration for the academic work with science and technology studies (STS) and ethics of technology (Grunwald, 2011;von Schomberg, 2012). The promotion of RRI at an EU-policy level creates a new context for developing participatory processes with the aim of bringing together diverse actors of research and innovation. While RRI is increasingly discussed in policy circles and by academics, these discussions are yet to be operationalized into practical guidelines for participatory processes. Still, the current debate on RRI can accommodate the discussion of aims of participatory processes, where the concept of responsiveness has emerged as one of the keys aspect for achieving responsibility (von Schomberg, 2011;Stilgoe et al., 2013;Fisher & Maricle, 2014). While the literature on RRI defines dialogue and deliberation as key ways to obtain responsiveness, it lacks guidelines for how to develop participatory processes that promote responsiveness.
It is therefore worth learning from existing literature discussing how to create dialogue across institutions and professional divides.

Creating Spaces for Dialogue Across Boundaries
The current interest in creating collaboration across institutional and disciplinary divides at an EU-policy level is also visible in the academic literature. Despite differences among scholars of deliberative democracy, "All see communication across difference as key to resolution …" (Dryzek & Niemeyer, 2010, p. 93). Focus is on the creation of spaces where such dialogue can take place, which can bring stakeholders together across disciplines and institutions (e.g. Dryzek, 2010, p. 167). This tendency is also visible in the governance of R&I. Nowotny et al. (2001) describe the need for an agora, a space that can create dialogue between science and society as a necessary step to re-imagine science and improve social inclusion. Jørgensen & Sørensen (1999) discuss the need for "Arenas of Development" to create spaces that foster innovation. Others discuss the need for boundary organizations (see Guston, 2001), organizations that can create spaces for cocreation as well as mediating among actors of R&I. The article's use of the term boundaries to describe the context of stakeholder workshops in R&I is inspired by earlier discussions of boundary work (Gieryn, 1983;Quick & Feldman, 2014).
Similarly to the argument of responsiveness in literature on RRI, this work argues that collaboration across divides can provide "the ability of public managers and others to use collaboration to reassemble resources and activities to continue addressing critical public problems despite disruption or adversity" (Quick & Feldman, 2014, p. 674). These theoretical contributions contribute to and reflect on policy developments in the governance of R&I, where experiments with knowledge platforms and foresight have attempted to establish new spaces for interaction among actors of R&I. Central to the success of such initiatives is the ability to create a constructive dialogue among the participating stakeholders.
The development of participatory processes for the governance of R&I is inspired by ideas of deliberative democracy. At the core of deliberative democracy is how decisions should be developed through: "… processes of judgement and preference formation and transformation within informed, respectful, and competent dialogue" (Dryzek, 2010, p. 3). The current focus on RRI adds a new perspective on the aim of participatory processes through its focus on responsiveness. Pellizzoni (2004, p. 559) argues, "The concept of responsiveness … allows a better evaluation of different 'participatory' processes and their significance as innovative forms of governance". Because the theoretical contributions to literature on RRI mainly focus on if and why we need further deliberation, the contribution seldom describes how to improve such deliberation through better participatory processes. However, research shows that the quality of the participatory process is essential for the process to live up to its potential for improving governance (Renn, 2008, p. 283).

Synthesizing Literature to Provide a New Perspective on Stakeholder Deliberation
The article contributes to literature on stakeholder deliberation by synthesizing three kinds of literature. First, to understand why responsiveness should be a main aim for deliberation in R&I, the article draws on recent literature of RRI (Fisher & Maricle, 2014;Nielsen, 2016;Pellizzoni, 2004;von Schomberg, 2011;Stilgoe et al., 2013). Second, the article draws on literature describing the ideals and values of participatory processes (Bussu et al, 2015;Callon, Lascoumes, & Barthe, 2009;Chilvers, 2008;Renn, 2008). At this overall level, the theoretical literature is concerned with the role of dialogue between groups in our political system. This literature discusses what role deliberation should play in politics and outlines the ideals that should guide such deliberation. Third, the article draws on handbook literature to integrate guidance on best practices for stakeholder participation (Brown & Isaacs, 2005;Gardner et al., 2009;Owen, 2008). At this more practical level, organizers of stakeholder workshops find inspiration in documentation of workshop experiences and the handbooks describing best practice. Connecting the theoretical ideals for dialogue with the challenging practice of organizing stakeholder workshops is an important step for further development. While there is a vast theoretical literature on dialogue, it has been questioned how often and thoroughly the approaches are applied to deal with multi-stakeholder dialogues (Payne & Calton, 2002), which is the context we want to examine further. Similarly, it is argued that despite a growing literature describing and discussing good practices of organizing stakeholder workshops, this literature lacks a strong theoretical foundation (Gardner, Dowd, Mason, & Ashworth, 2009, p. 5). There is therefore a need to better connect the relevant literature on stakeholder workshops to the current work on deliberation in the multi-stakeholder context of R&I. In looking at workshop initiatives involving multiple stakeholders, Payne and Calton (2004) argue, "[h]ow to initiate this dialogic model of action more effectively and ethically seems to deserve much greater research and experimentation" (p. 76).
The article is structured in two main sections; the first discusses how existing literature on stakeholder workshops can enlighten the current work with developing responsiveness among actors of R&I, while the second illustrates the challenges for operationalizing theory in practice. The illustration is based on a recent experimentation with deliberation among actors of R&I in five European stakeholder workshops concerning RRI. Both sections are divided into three parts, structured according to three steps in the workshop process, which will be explained below: 1) planning and design; 2) workshop interaction; and 3) the gathering of conclusions. Finally, the conclusions and their consequences for the relationship between theory and practice will be discussed.

Achieving Responsiveness Among Actors Through Stakeholder Workshops
In the following section, we will examine how stakeholder workshops can be conducted to support responsiveness among actors building on both theoretical literature and handbook literature on stakeholder workshops. While the theoretical literature discusses the ideals of participatory processes, the handbook literature can contribute experiences and lessons from practice. The section will focus on three phases in the workshop process: creating the framing conditions, designing and facilitating the workshops, and collecting the conclusions, as well as procedural lessons. An unlimited number of different types of workshops exist, each with its own form and purpose. Every aspect of the workshop is affected by the aim and context. The article highlights aspects that are relevant to the five European workshops and have a general importance for theory on stakeholder workshops. The figure sums up the conclusions, which will be discussed in the following sections.

Creating the Framing Conditions for Dialogue
The first phase of a stakeholder workshop focuses on deciding the aim, content, and design, and defining the relevant stakeholders. This phase includes a comprehensive recruitment process, which makes sure the design correlates with the stakeholders interested in attending. The main challenge of the first phase is creating the right conditions for fruitful participation. For responsiveness to be in focus, making room for learning and adaptation must be considered.
The first step of a stakeholder workshop is to define the overall aim for the process. Fiorino (1990) argues that there are three arguments for participation: first, that participation can improve processes, an instrumental argument; second, that participation improves the final product by increasing knowledge, a substantive argument; third, that relevant groups have a right to participate and be heard, a normative argument. While all three arguments for participation are part of most participatory processes, it is the instrumental arguments that emphasize

Creating the framing conditions for dialogue
• The objectives of stakeholder participation focus on understanding the positions of other stakeholders and making transformation and joint action possible. • Real engagement requires participants who are interested and committed to the dialogue. • While representation of a dialogue is important, participatory processes should be improving the representativeness of the policy process as a whole. • There needs to be alignment between theme, information, and participants of the workshop.

Organizing workshop interaction
• Workshops should find a balanced way to include self-interests. The interest and perspectives of participants should be included, while also making room for learning new perspectives. • Disagreement is essential for learning, and there should also be room for more emotional disagreement. However, a real conflict can block further dialogue between participants and should be avoided. • The opposite of an emotional debate is one where participants are not engaged. Creating engagement is essential. Giving the participants ownership of the process is key for them to feel that their work shapes the process. • Good questions can help to open up discussion and reflection and probe whether relevant perspectives are taken into account in the argumentation.

Collecting the conclusions and procedural lessons
• Both the agreements and disagreements of the group should be documented and supplement the work for consensus. • It is an important balance to make sure that the results reflect the workshop discussion, while also giving a clear message to outside target groups that participants aim to influence. • Organizers will need to choose the perspective(s) from which to evaluate the process, or potentially have several types of evaluation, as every participant will participate in the workshop for different reasons.
responsiveness and learning among actors. Such instrumental aims are often emphasized in theory of stakeholder deliberation. Mansbridge (2003, p. 179) suggests two overall objectives for deliberation based on the current theoretical literature: Creating awareness of preferences (both one's own and those of others); and creating conscious transformation of interests. While these objectives focus on the outcomes of deliberative processes, they stand in contrast to a narrow focus on improving the content of decisions, the substantial argument. An instrumental approach therefore demands that the organizers of a participatory process create space for reflection, which the creation of awareness and change requires. What makes dialogue special is the way it can take us away from our individual context and preconceptions and create new ways to reflect on issues. In a dialogue, there are parallel processes of trying to find unity and acceptance, and processes of distinction and separation (Shotter in McNamee & Shotter, 2003, p. 99), which both contribute to such reflection. Another objective that is suggested for participatory processes is to make collective action possible through a common understanding of problems and potential solutions, which is sometimes referred to as dialogue for problem-solving (Pauly, 2003, p. 254). Theories can, therefore, help us define diverse potentials of participatory processes, which need to be further prioritized and refined for any specific workshops.
Deciding on the specific content of a workshop is another challenge in the first phase. One handbook suggests that the problem taken up in the workshop should be context specific: "Presenting stakeholders with a practical and locally-relevant problem will draw more attention and foster a greater sense of involvement than asking them to consider a general topic" (Gardner et al., 2009, p. 6). There needs to be the well-considered correlation between the aim, content and the participants of the workshop for it to be fruitful. Good dialogue begins with getting the right people together, although this is no easy task. According to Owen (2008), dialogue is at its best when only the people wanting to be there are present. However, in stakeholder processes with specific aims, it is often necessary to have certain perspectives included, even if specific stakeholders might need more persuasion than others to attend. Diversity is, therefore, an important value in stakeholder dialogue (see Brown & Isaacs, 2005). This way of understanding participatory processes emphasizes the idea that participatory processes should extend the representation of existing decision-making processes. It merges the ideas of representative democracy, which emphasizes the representation of relevant groups, with the ideals of participation in a direct/pure democracy model. Theorists see participatory methods as an extension for improving and supplementing a lack of representativeness in representative democracy by creating continuous interaction between those represented and the representatives (Bohman, 1998;Brown, 2009;Renn, 2008, p. 309). Participatory processes can supplement representative processes by creating diversity in participants, thereby making sure that both ideas of representativeness as well as inclusion have been taken into account (Smith & Wales, 2000). In creating the framing conditions for stakeholder workshops, one should therefore consider both the aims of stakeholder dialogue and the importance of participation contributing to existing representative processes.
The idea of responsiveness emphasizes learning processes among stakeholders, in addition to improving links from stakeholders to decision-makers. It therefore emphasizes that a participatory process should not only produce results for policy, but also create an environment where participants can develop awareness and understanding of the others' positions. The following four points should be considered: • The objective of stakeholder participation can be put into a first-level objective of understanding the other positions and a second-level objective of making transformation and joint action possible. Both objectives are closely related to the idea of responsiveness. • Real engagement demands participants who are interested and committed to the dialogue. Recruitment should also take into account that there should be a mutual interest in debating and developing action on a topic. • Participatory processes are increasingly supplementing and improving existing policy processes. Actively engaging groups that are not part of existing policy processes can be a way to improve overall representation. • Questions and information used in the workshops should be relevant to participants. In other words, there needs to be a good alignment between theme, information, and participants.

Organizing Workshop Interaction
For the workshop to facilitate responsiveness, the participants need to be committed to the process and attentive to the perspectives of other participants. The main challenge for the actual workshop process is to get participants engaged, while making sure that relevant perspectives are listened to and adequately considered.
The theoretical literature tends to make a sharp distinction between processes building consensus through negotiation of interests and deliberative processes to open up dialogue (Kerkhof, 2006); yet, in praxis, both processes co-exist (see Phillips, 2011, p. 52). Kaner (2014) describes this as co-existing processes of convergent and divergent thinking. According to Mansbridge (2003, p. 182), legitimate perspectives are obstructed if we see the inclusion of self-interests, and the bargaining that comes along with it, as in opposition to dialogue and deliberation. Rarely would situations have one common good around which a consensus can form, because there are legitimate differences of interests.
Consequently, any deliberation should include both articulations of the common good and narrow self-interests.
One area in which literature on deliberative democracy deals unsatisfactorily is the area of emotions (Sanders, 1997). As described by Marcus (2010, p. 7), "Reason is commonly portrayed as a fragile force for progress, justice, and greater democracy, which requires protection against the intrusive and destructive impulse of emotion." However, dealing with emotions is an unavoidable part of dialogic practices. Not allowing such emotions, because of fear that debate might become irrational, would potentially remove important expressions of injustice (Mansbridge, 2003, p. 188).
The space for dialogue should therefore provide room for emotional statements, without such statements dominating. A similar balance is needed when dealing with conflicts: "Good deliberation … requires trying to move toward consensus while retaining and refusing to downplay or suppress existing elements of genuine conflict, either in opinion or in interests" (Mansbridge, 2003, p. 189). Subsequently, what is needed is not a completely rational debate, but some sense of "mutual respect" (Mansbridge, 2003, p. 190) or "reasonableness" (Young, 2002, p. 24). While making disagreements a natural part of the workshop, the disagreements should not create barriers between participants, thus blocking any further deliberation.
While the theory points to the challenges of self-interests and emotions, a central challenge described in the handbooks on dialogue is commitment and ownership. The starting point of a dialogue is finding ways where everyone can contribute.
Providing the workshop participants with information through a presentation tends to create some kind of reflection and questions, but seldom an actual dialogue. For a dialogue to be created, there is a need to activate workshop participants as early as possible. Often this activation is begun by proposing a question. The power of good questions to initiate dialogue has been described as follows 2 : "Good questions help us become both curious and uncertain, and this is always the road that opens us to the surprise of new insight" (Wheatley, 2005, p. xi). As important as getting the participants activated early is to have them feel some level of ownership for more than just their own contribution. Ownership of the process can, for example, be created by making participants representatives of work done in an earlier group: "Each time we move to a new table, we lose more of ourselves and become biggerwe now represent a conversation that happened among several people" (Wheatley, 2005, p. xi).
Central to the workshop process is the effort to give participants a level of ownership that ensures that they feel responsible and engaged in developing the results. The discussions among participants should reflect a respect for all participants to contribute, while also giving room for disagreement. The following should be considered: • A dialogue workshop is not about putting self-interests completely aside, nor should it be about the simple promotion of self-interests that can take place in a negotiation. A good workshop can include the interest and perspectives of participants while also making sure that participants are open to learning about other perspectives. • Similarly, a balance needs to be struck when it comes to handling disagreement. Disagreement is essential for learning, and there should also be room for emotional disagreement. However, a real conflict can block further dialogue between participants and should be avoided. • The opposite of an emotional debate is one where participants are not engaged. Creating engagement is essential. Giving the participants ownership of the process is key for them to feel that their arguments help shape the process and not the other way around. • Finally, good questions can help open up discussion and reflection.
Questions can start a debate and probe whether relevant perspectives are taken into account in the argumentation.

Collecting the Conclusions and Procedural Lessons
In the final phase, the results of the workshop have to be collected. While the second phase aims to bring out both agreements and disagreements among the participants, the third phase is focused on capturing the diversity. Responsiveness in this phase demands that the final results are attentive to the input given during the process.
A central discussion in theory on dialogue is whether there is a clear endpoint toward which to work. Habermas (1985) describes his ideal speech situation in which it is possible to have a free, open negotiation of differences. By establishing this ideal, Habermas wishes to discuss the conditions in which such ideal speech can take place, with the aim to uncover the power relations that create barriers for dialogue. While the ideas of Habermas still provide the foundation of most theories of deliberative democracy, they have also received substantial criticism. One of the most extensive and well-recognized criticisms comes from Mouffe. Mouffe (1999) argues that while the objectives of deliberative democracy-reducing the role of economic interests in the public sphere-are praiseworthy, Habermas' ideas of consensus processes do not provide a worthy alternative. For Mouffe, dialogue should be concerned with finding good, but temporary, compromises, while giving room for the inclusion of all perspectives. While a temporary consensus is not necessarily a negative, it should be emphasized that consensus always reflects an active political decision and can never be a neutral and objective description of the common good.
Mouffe's critique of consensus, positing that it can help hide important political differences, has inspired a number of practical approaches to harnessing the results of dialogue. Clearly, there is an important balance in any dialogue between opening up new thoughts and perspectives and closing down to form common agreements and decisions (Stirling, 2008). Following the line of thinking that points out that disagreement can be as informative as consensus, it has been suggested that a "deliberative mapping" of the diverse arguments represented in a group should be conducted (Burgess et al., 2007). A good dialogue should include considerations of both Habermas and Mouffe as "participatory approaches always combine some degree of consensus-oriented cooperation and some degree of compromise-oriented negotiation" (van den Hove, 2006, p. 13).
Another central discussion in the theoretical literature is whether it is in the process or the outcomes that dialogue finds its main purpose and results. Mansbridge (2003, p. 180) underlines that "the failure to reach consensus on a just or good outcome does not automatically mark a bad process of deliberation." Theories tend to emphasize the representation of the relevant groups in a fair manner as a main criteria of success for deliberative processes (see Renn, 2008, p. 274). However, this is in contrast to literature looking at the aims of dialogue according to the participants, which tends to emphasize substantial outcomes. A study looking at how participants in a dialogue would evaluate it emphasized "strong" outcomes and good dissemination to achieve maximum impact as central objectives (Oels, 2007). Theory on deliberative democracy focuses mainly on the quality of the process, while theory based closely on practices of stakeholder dialogue tend to emphasize that dialogue is instrumental to decision-making processes (see also Hennen et al., 2004).
Evaluation is a final important aspect of a stakeholder workshop. Participatory processes can have multiple aims and be approached from different points of view. Any evaluation of stakeholder workshops needs to take this into consideration. While some argue that an overall framework for evaluation can be made (e.g. Burgess & Chilvers, 2006), others argue that dialogue processes stem from separate paradigms, each with its own understanding of aims and evaluation criteria (e.g. Renn, 2008). Both views see the inclusion of context and aims of the workshop as crucial for creating a relevant evaluation of stakeholder dialogue, while also describing a number of key dimensions for creating constructive dialogue. An alternative approach is to let participants decide evaluation criteria (Oels, 2007).
However, participants will also have different reasons for participating and, accordingly, different aims of evaluation, which again will differ from the objectives of the organizers.
Being true to the idea of responsiveness also means listening to all groups that are involved. Working toward consensus can be part of this process; yet, it is important that all aspects are brought forward and considered. Finding ways to document the results capturing the diversity of the group is central to the process. Three elements should be considered: • Organizers should therefore find ways to work toward consensus, documenting the disagreements in the process. • While a good and fair process is important, the need for creating "strong outcomes" should not be underestimated. It is an important balance to make sure that the results fairly reflect the workshop discussion, while also giving a clear message to outside target groups that participants feel they have influenced and can benefit from the process. • A final challenge is to find appropriate ways to evaluate the workshop.
Organizers will need to choose the perspective(s) from which to evaluate the process, or potentially have several types of evaluation. As every participant will participate in the workshop for different reasons, which again might be different from the organizer's reason for initiating the workshop, each of their perspectives will give different evaluation criteria.

Illustration of the Challenges and Incremental Learning of Organizing Stakeholder Workshops
So far, the article has discussed the theoretical foundation for organizing stakeholder workshops. In the following illustration, key elements of the discussion will be examined against the practice of designing five European stakeholder workshops. The challenges discussed will be recognizable for all actors organizing multi-stakeholder workshops in areas including diverse and specialized forms of expertise. The illustration will describe how the challenges of improving stakeholder dialogue across boundaries in R&I were addressed by the Res-AGorA team. In organizing five European workshops on responsible research and innovation (RRI), the Res-AGorA team aimed to engage not only scientists in dialogue but all stakeholders relevant to the governance of research and innovation (R&I). Dialogue is a challenging and, by its very nature, open-ended process that is difficult to manage. No dialogue is perfect, yet many of us have experienced its potential transformative value.
The illustration is based on observations, notes, as well as oral and written evaluations of five stakeholder workshops concerning the governance of RRI, which were organized in the spring of 2015 as part of the EU Res-AGorA project. The EU project aimed at creating a governance framework for RRI (Lindner et al., 2016). Each of the five workshops was held over two days and co-organized by a team of partners 3 , with the Danish Board of Technology (DBT) as the lead. DBT has been internationally recognized for its role in developing participatory methods (Andersen & Jaeger, 1999;Horst, 2014;Sykes & Macnaghten, 2013). DBT aims to develop structured and innovative dialogue processes, and the illustration is seen from this perspective. The detailed facilitation 4 , dissemination, and follow-up of the results lie outside the scope of the illustration. It is not an evaluation of the design or the workshops themselves, but an inspiration for discussing and experimenting with dialogue challenges in practice. Studying a phenomenon from an insider perspective can provide a detailed understanding of practices as the researcher has knowledge of the context. However, it also brings about questions of objectivity and authenticity (Dwyer & Buckle, 2009). By taking an inside perspective, we aim to provide guidance based on knowledge of existing practices.

Creating the Right Framing Conditions for Dialogue
The Res-AGorA workshops aimed at creating a generic framework for engaging stakeholders of R&I on issues relating to RRI. The workshop was a chance to learn from participants about how they engage stakeholders, while also being an experiment in carrying out such a stakeholder process. Participatory processes should, according to theory, contribute to raising awareness, changing interests, and finding common ground for problem-solving. The aims of the Res-AGorA workshops include these goals in different ways. First, the exchange of experiences should raise awareness of RRI for both stakeholders and members of the project. Second, the project was testing out ways to build platforms for future dialogues, which could inspire all participants.
The Res-AGorA team aimed to involve participants who had expertise in the topic in addition to interest and capabilities in building bridges across institutions. The challenge of recruitment was balancing the team's idea of a diverse and relevant 3 The team included The Institute for Advanced Studies (IHS), Fraunhofer ISI, and University of Twente. 4 For a detailed description of the facilitation of stakeholder workshops, see e.g. Kaner (2014).
The workshops tested the generic workshop framework on five different issues relating to RRI. Each workshop included between 15 and 25 participants from research institutions, government, foundations, companies, international organizations, and NGOs. group of participants with finding participants who would be committed to the process. In the recruitment process for the workshops, telephone interviews played an important role in picking out relevant stakeholders and for identifying the right framing of the workshops. The interviews were conducted in addition to the desktop research and e-mail contact with potential participants. The recruitment process is challenging as no handbook can tell whom to invite, but over time a better picture of the advantages and disadvantages of inviting certain stakeholders or individuals develops. Each workshop will have its own set of questions to define the most relevant individuals. In our case, those already playing a formal representative role were more likely to participate, but it was important for the team to make sure that the final group also included participants who did not normally take part in existing decision-making procedures.
Subsequently, recruitment was about representativeness and diversity, aiming to include participants who can bring new perspectives and are committed to the process.
The overall topic of the workshops was RRI. While the theme of RRI ran through all the workshops, each workshop also had a more specific theme and target audience. This approach was taken to make sure that participants would find discussions relevant and were able to contribute based on their practical experiences. The workshops were designed for the specific context, participants, and aims of the Res-AGorA. The main aim was to receive input on the Res-AGorA governance framework for RRI. Secondary aims included disseminating the preliminary results of the project and playing a role in building a community for promoting the discussion of RRI. Designing interdisciplinary workshops presented challenges. First, the written background material had to be understood across different disciplines, thus creating a basis for a discussion where everyone could participate. Therefore, significant time was put in trying to "translate" the project material. Second, this "translation" was also needed in the workshops; consequently, the recruitment prioritized recruiting participants who had experiences in working across disciplines and institutions.
The first phase can shape the conditions for promoting responsiveness in the workshop. This phase includes significant challenges in adapting the design and finding the right participants. Not all the stakeholder groups will be equally interested in participating. In our case, it was challenging to recruit politicians and stakeholders from industry and, therefore, the recruitment process had a special focus on these groups in the later stages. The final group of participants represented a diverse selection of the R&I stakeholders, while also including a number of actors who have key roles as bridge builders between institutions. Creating material for the workshop that fit the whole group was a major challenge as understanding of the topic as well as language skills was highly varied.

Organizing Workshop Interaction
In general, conversation at the workshops was abundant, but it also included misunderstandings and occasional awkward silences. While one might think that interaction across institutions and disciplinary boundaries happens regularly, for many of the participants, this was not just a usual day at the job. Participants were constantly interpreting conversation into their own context, which might be difficult to follow for other participants. Project participants faced similar challenges. Before the workshop, extensive empirical and analytical work had been carried out, so the main conclusions could be presented. Now, the work was interpreted anew by the workshop participants. For participants to take responsibility for the development of ideas, they need to feel some level of ownership of the workshop process, which was an important challenge. Small changes to the design, creating a clearer structure for the workshops, helped improve this element between the five workshops.
During the workshops, it became clear that participants had different ideas of their own roles and how they were to represent their organizations. A key objective of dialogue is to create awareness about preferences; however, some participants took part primarily to represent their organization and its members. These participants were primarily there on behalf of others and skeptical of exploring their personal preferences regarding the topic. Such a literal understanding of representation creates a barrier for learning and responsiveness, which should be considered for stakeholder workshops. While specific interests and strong opinions are not generally a problem, in certain cases they do hinder dialogue because people stop listening. Listening was, therefore, a key component of the "rules of dialogue" that were sent to participants ahead of the workshops. When being moved to a new table, participants became "rapporteurs" who had to communicate earlier conversations to the new group, which created positive dynamics as participants tried to represent and explain arguments from the former group in a fair way.
Some of the topics debated were strongly contested, creating an emotional debate among participants. As organizers, we debated whether to interrupt heated arguments in order to bring the discussion back to the core questions. However, when the facilitator did not interfere, participants took on the responsibility to bring the debate back on track. This way of handling conflict seemed like a good compromise that made sure there was room to air frustrations while not letting the The workshops consisted of the following phases: 1. Exploring stakeholders' experiences with RRI 2. Presenting dimensions and principles of RRI 3. Making effective use of The Navigator's* dimensions and principles 4. Effectively practicing RRI *Read about The Responsibility Navigator here: http://responsibility-navigator.eu/navigator/ frustrations take over the process. It is a fine line to know when to let the participants steer, thereby building responsibility for the process and results, and when the facilitator should step in. The participants of the Res-AGorA workshops were used to participating in deliberations, although not with this type of diverse group, and used to expressing the self-interests of their stakeholder perspective. What was most important, as seen from the organizers' perspective, was that the setting allowed for the inclusion of all relevant perspectives, whether they included self-interests or emotions.
The second phase concerns the interaction among participants and therefore presents challenges for how to create responsiveness in the workshop process. The design aimed at creating discussion and having people reflect on their own positions as well as other people's perspectives. During the process, the design was adjusted several times to adapt to changes suggested by participants and organizers. Having five workshops with which to test the design using different participants and in different settings helped improve the design significantly. Several changes were made after the first workshop, as the progress through the different sessions was unclear for participants. In addition, fine-tuning was happening both after and during each workshop to adapt to context and unexpected circumstances. The evaluations contained both critique of a lack of steering as well as over steering, showing the difficulty in finding the right balance. Not all of these stakeholders were equally committed to participating, and some found the materials, as well as language, used at the workshop difficult.

Collecting the Conclusions as Well as Procedural Lessons
While opening up dialogue was an essential objective of the Res-AGorA workshops, equally important was gathering concrete feedback on the work of the project and possible paths forward. Making sure that the conversation progresses was not only important to the project, but also for participants to feel that the workshop was dynamic and moving forward. It was decided that there was a need for two types of note-taking to document progress. The participants made their own notes on flip charts, which they could use to document and share the conversation at the table. These notes contained the participants' own summaries of the sessions. When participants were told to move tables, one participant stayed at the table to present the work of the former group. To ensure that no points were lost, a member of the Res-AGorA consortium took additional notes at each table. These notes were not part of the shared conversation, but were made to ensure that important points were not lost for the projects' further development. These notes could be more comprehensive, including smaller points Read more about the co-construction process developed in the project and who can apply it here: http://responsibilitynavigator.eu/co-construction-method/ that might not make it onto the flip charts of the shared notes for the group.
The multiple aims of the Res-AGorA project also demanded different forms of evaluation. Central to the evaluation was the procedural learning process, meaning continuously improving the design of the workshops. There was continuous evaluation both among the project participants organizing the workshop and with the participants of the workshops. Oral evaluations, both with participants and among the project members, made sure that the design could be adapted continuously. A written evaluation gathered the more general points and made it possible to compare across the five workshops. The focus on the evaluation of process meant less focus on evaluating the content and substantial outcomes of the workshops.
In the third and final phase, the workshop aimed to show that the results are responsive to the input of participants. For the Res-AGorA workshops, there was no objective of consensus or specific decisions that had to be taken; yet, for the participants, it was natural to look for agreements and alliances. The job of the organizers was to create a space that welcomed both disagreement and agreement, as well as to document the progress of the dialogue toward a mutual understanding. The different kinds of notes helped to collect lessons throughout the workshop process and were all used for the final workshop report.

Conclusions -Promoting Responsiveness Among Actors Through Stakeholder Workshops
The article began by introducing the idea of RRI and the objective of improving responsiveness among actors of R&I. The article then discussed three phases of the organization of stakeholder workshops. Central to the first phase is ensuring diversity of participants in the recruitment process, as stakeholder dialogues aim to bring in new perspectives to governance processes. Finding the right participants and making sure the content is relevant is key in this phase. In the Res-AGorA project, phone interviews with potential participants were used to qualify both recruitment of participants and the content for discussions. The second phase concerned the workshop interaction. Here, the theoretical literature emphasizes challenges relating to self-interest and emotions that need to be balanced in the workshop. While these are definitely themes to consider, they were not major challenges in the workshops held by Res-AGorA. More important was the challenge of opening up a good dialogue through asking the right questions and finding ways to give participants ownership of the process. These topics are taken up and discussed in the stakeholder workshop handbooks that inspired Res-AGorA. Third, the article described the challenge of finding good and appropriate ways to harness the results. The theoretical discussions focus on the alternative between processes attempting to reach a consensus and processes highlighting difference in opinion, but a dialogue will always include both processes. By documenting the process as well as the results, the Res-AGorA workshops tried to capture both diversity and consensus. The workshops further showed that there is an unfilled potential for dialogue among actors in R&I. It was clear from the workshops that for most participants, working across disciplinary and institutional divides is a rare activity. There is a need to build capacities for working across boundaries as a starting point for improving interaction.
Theoretical perspectives on stakeholder workshops provide limited answers as to how to prioritize limited resources when trying to achieve multiple aims. Nonetheless, theory can provide challenging questions that can help the constant re-evaluation of best-practice for stakeholder dialogue. In this way, the most fruitful relationship between theory and praxis might be one of cross-pollination rather than the creation of one common framework. A fruitful relationship between theory and practice demands sensitivity and interaction to find creative solutions to both small and large challenges. The main strength and challenge of dialogue processes is the openness and unpredictability of the space created. No two dialogues are the same; yet, over time we can learn to sense when spaces are creative and when they need new stimulations. Such practical knowledge can never be fully theorized but must be recognized as a necessary condition for improving the organization of stakeholder workshops.