Evaluating knowledge transfer at the interface between science and society

Societal challenges are increasing on a global scale, requiring an intensification of knowledge transfer(KT)between scientific and societal actors on multiple levels. These transfers bring up new demands regarding the way in which knowledge is produced and transferred, which mechanisms are utilized to ensure the quality of these knowledge interactions, and how such interactions can be evaluated. Capturing and evaluating KT, however, also opens up new reflections about what a meaningful “impact” is and how KTs are shaped and driven. The results presented here were reached by means of a formative and summative evaluation approach, and include an accompanying research effort that aimed to capture central KT processes from start to “finish” by a tailored KT process assessment framework. This framework was applied to the ongoing activities of twelve in-house KT projects (from 2014 to 2017) that were conducted in a fundamental natural science research institution within the field of earth system science. Our findings indicate that, among other things, the continuous assessment of the underlying processes of KT allows for processes and outcomes to be directly influenced, while also providing scope for institutional learning. Better insights into the definition of “societal relevance” may in fact not start with the result, but rather with the question of how the research will be conducted and for whom.


Science and its role in addressing the grand challenges
The United Nations Sustainable Development Goals (SDGs) call on scientists and researchers to tackle society's grand challenges (Ober gassel et al. 2017). Society largely trusts and expects science to inform the debate on major issues and address the relevant social and environmental challenges. Research institutions are increasingly embedded in a social environment, which values research but questions more and more which research ventures are pursued, how knowledge is produced and transferred, and what mechanisms are utilized in order to control the quality of knowledge transfer (KT) (Lang et al. 2012, Barker and Kitcher 2014, Pohl et al. 2017. To this end, new relevant tools to evaluate KT efforts need to be established in order to meet changing requirements. Analyses of interdisciplinary (Fazey et al. 2014) and broader research (Wilsdon 2015) find that across different research communities the description, production and use of indicators remains contested and open to misunderstandings. Therefore, Wilsdon (2015) concluded that quantitative indicators alone do not accurately depict KT impact. Thus, we have chosen the following question as the starting point for our paper: how can science demonstrate its efforts to tackle the grand challenges and engage in meaningful science-stakeholder dialogue.
We present a process assessment framework which closely followed twelve small stakeholder-focused research projects at the German Alfred Wegener Institute Helmholtz Centre for Polar and Marine Research (AWI), and the qualitative data gathered using it. The nature of the framework is self-reflexive and conceptual in order to allow for easy adaptation and modification by researchers in different disciplines or funding agencies. Applying the framework in practice by employing the grounded theory approach (Glas er and Strauss 1967) resulted in the formulation of four principles for good KT processes. From this, we postulate a set of general conclusions, which are intended to inform KT practitioners as well as accompanying researchers and stir up discussion around the processes underlying KT activities.

> Capturing and evaluating knowledge transfer processes Summative and formative evaluation
This study combines two central evaluation approaches. Summative evaluations aim to provide validation at the end of the project, for example, by counting the number of peer-reviewed articles and their citation frequency, while formative evaluation aims to enhance reflections to improve and refine project activities during a project's lifetime (Fazey et al. 2014).
We worked from the premise that the achieved "impact" of a KT-focused research project is the sum of all products and processes generated during the lifetime of the project. In order to develop a meaningful KT product a series of KT processes must be enacted, which entail a suite of specific, target-oriented activites. Simply put, a process is the "definition of the tasks and the sequence of those tasks necessary to fulfil an objective" (Davis 2009, p. 1). Central for successful KT is therefore the question, what type of impulses were initiated by these processes, and how and in what ways they have been coupled internally and externally.
The tasks, activities, or impulses are shaped by principle normative values that mirror the societal perception and expectation on how KT is conducted. If a good dialogue and feedback process reveals and deals with these underlying values in a given sciencestakeholder interaction project, the resulting products and impacts can be expected to be positive.
With this in mind, we developed and tested our KT process assessment framework (table 1, p. 286) by applying it to the ongoing activities of twelve in-house KT projects (2014 to 2017) that were conducted under the umbrella of the Earth System Knowledge Platform (ESKP) of the Helmholtz Association. These projects ranged ac cord ing to their primary methodological focus from 1. dialogue formats (i.e., on regional awareness of sea-level rise effects and climate change effects on local fisheries), to 2. data products (i. e., on the development of the sea-ice portal and tackling marine litter by litterbase) (figure 1) to 3. modelling (i. e., streamflow forecasting and extreme climate and weather). A detailed overview of these KT projects, their set-up and central processes is provided in Krause (2018).
Prior to the application of the KT process assessment framework, various workshops were conducted across the institute to understand what scientists defined as KT and what were institutionally regarded "positive outcomes". These workshops formed the basis for the pre-assessment criteria that mirrored the not yet explicitly defined institutional ideas on what constitutes a good KT process. The pre-assessment and selection of KT projects was conducted by a group of representatives from multiple levels within the AWI, ranging from the board of directors, senior scientists and research section heads, to representatives from different natural science disciplines.
Out of an array of diverse KT project proposals, six projects per year (2014 and 2015) were funded. Each KT project comprised on average a team of three to six fundamental natural scientists, all of which were subject to an accompanying formative and summative evaluation process during the entire life cycle of the respective KT project (table 1). The study originally set out to capture the entire KT process from start to "finish", capturing processes happening at the research institution as well as amongst the different targeted stakeholder groups. However, it was soon realized that the complexity of the KT process sharply increases once the information leaves the direct realm of science. The scope of the project was therefore limited to the parts of the puzzle that can be directly influenced from within the research and project management process, meaning the involved researchers. The first six projects underwent a pre-test of the KT process assessment framework that helped to test and adapt the methods and central questions during the different stages of the process evaluation. In the following, we showcase the results of the second group of KT projects (n=6) that underwent the entire KT process assessment cycle.
The ex ante assessment was conducted within the first eight weeks after a project had started. The assessment consisted of a standardized questionnaire which prompted self-defined output, potential indicators, individual self-reflection and a definition of key stakeholders.
Shortly after the halfway point of the KT projects lifetime the in itinere assessment was conducted for all projects. This phase consisted of a one-hour to 1.5-hour semi-structured interview with the scientists engaged in each of the KT projects. Hereby the status quo of the project, whether all actions and predefined outputs were progressing on time, if new outputs were discovered, with resulting new or more applicable success indicators, as well as any occurring challenges were addressed. These were then discussed jointly with the process assessment team to support the respective KT project in their further development by external review. Answers to interview questions were collected and later collated by the interviewers according to McLellan et al. (2003).
The ex post assessment was conducted approximately one month after the projects' completion. Quantitative-qualitative mix methods (standardized questionnaire and semi-structured interviews) were applied to gather information on all final outputs as well as final suggestions for success indicators of KT. Similarly, the ex post + 1 year assessment was conducted employing semistructured interviews in which the further development of outputs were discussed.
In the following, we describe and collate the central findings along three major issues of relevance for KT processes: 1. outputs and indicators of success, 2. self-expectation/perception of the KT scientist, and 3. knowledge on key stakeholders. 1: Stages of the knowledge transfer (KT) process assessment framework, criteria and used methods and their central rationale during the entire project life cycle. Pre-assessment was conducted by an internal interdisciplinary expert panel that identified those projects that where deemed to hold sufficient KT potential. Ex ante stage was carried out within the first two months of the selected project, in itinere after the project passed its half waypoint and the ex post stage was carried out after the last month (resp. after one year) of the project. Each stage is followed by a reflective turn, giving both participating researchers and evaluators the opportunity to assess their methodology before moving forward to the next project phase.

> Outputs and indicators of success
One of the core pillars and objectives of applying the KT framework was to tackle the issue of how to capture "success" of KT.
To this end, the partaking researchers were asked in the ex ante stage to predefine the planned outputs of the project, applicable success indicators as well as to identify potential key stakeholders of the project. Most of the potential output types identified at this early stage were based on the strong product-oriented understanding of KT within the institute's research agenda. A total of 28 different outputs and corresponding sets of indicators were defined across all projects. The number of outputs varied across proposed output categories with the category "interview" and "advice to stakeholders" not being chosen at all at this point (figure 2, left). The most common planned output category chosen were, in order, "other" (i.e., blog, poster, infographic, etc.), "new or improved product, process or service" and "peer-reviewed article". Furthermore, the differentiation between method and output was often found to be skewed. For example, science-stakeholder workshops and the knowledge generated through them were, at first, not regarded as attributable central KT output. During the in itinere progress assessment stage, a formal first test trial of the previously self-defined success indicators of outputs was conducted verbally with participants and perceived as helpful to detect and reflect whether they adequately captured the relevant KT processes inside the project. This test was conducted by having participants score the status of their outputs from "not met", "initiated", "on time", "completed" to "completed and expanded". However, the quantification of the predefined metrics proved in places to be difficult for the respective project participants, affirming the ongoing discussion on the need to include qualitative data in the assessment of KT (BEIS 2016).
In the ex post stage, the originally developed criteria of success of project-specific outputs were revisited and ranked to their degree of fulfilment on a scale from 0 ("not achieved") to 5 ("fully achieved"). The ranking provided a quantifiable degree of achievement for each output while also providing insights into the degree of applicability of the respective indicator. A total 32 outputs were produced by the six assessed projects up until the ex post assessment. In table 2 (p. 288) some of these are exemplified.
This number is very close to the amount of planned outputs during the ex ante stage but stands in stark contrast to the total number of outputs (82) produced when the ex post assessment was repeated after a year (figure 2, right). This observable increase reinforces the call for process-related accompanying research which starts simultaneously with the project, but runs longer than the other projects to allow for completion of synthesis building (Defila and Di Giulio 2018).
Quintessential outputs of KT, understood here as co-produced knowledge in close dialogue with stakeholders (Spaapen and van Drooge 2011), were often not considered as such, and thus overlooked and not attributed as successful results by the scientists involved in a given project. This was in part due to the somewhat undifferentiated offered output categories in the initial ex ante survey, which did not match well with the plurality that KT process-es entail. Thus, institutional learning took place: it was realized across the different partaking scientists and disciplines that preset typologies of output product categories do not reflect KT. This indicates that new output categories are needed to convey the plurali ty of meaningful KT outputs and better capture the often dialogic nature of KT projects. However, despite the recognized limits of such preset typologies, these categories were demanded for by the scientists and hence were kept throughout the entire KT pro cess evaluation.

Self-perception of the knowledge transfer scientist
The second part of the standardized questionnaire tried to gain an overview of the plurality of impacted dimensions while also encouraging self-reflection of the partaking scientist within a KT project. On a forced choice scale, ranging from 0 ("lowest") to 5 ("highest"), researchers were asked to rate their personal expectation in regard to the likely impact of their research. To this end, we provide ten potential dimension categories, based on the possible dimensions of scientific impact as defined by Godin and Doré (2005). The categories were presented as term without further definitions in order to capture respective contextual particularities of the different assessed projects, as well as the underlying principle normative values of the individual scientist in first approximation.
In the beginning (ex ante) the sum of the personal impact perceptions across all project members revealed a bias towards the expectation of creating impact in the dimension of environment, followed by science, policy, economy and society (figure 3, p. 289, left). This observation is not surprising, as scientists are engaged and evaluated primarily by these dimensions on a regular basis, follow ing the "publish or perish" dictum. Thus, it is an indication of an existing gap of knowledge in the realm of fundamental (natural) research on how KT projects differ in their scope and intention in contrast to the "classical" research impact efforts. During the ex post stage, the different categories of perceived impact were re-evaluated by the researchers (figure 3, right). The results showed a shift in the perceived impacts, positively towards education and technology and negatively steering away from economy and environment, indicating a learning process on the individual level in the sense that the engagement with stakeholders led to topical shifts, that is, of a project originally expected to influence economy shifting to technology impacts; a project originally expected to influence policy moving to society, etc.

Knowledge on key stakeholders
The last part of the standardized questionnaire asked for detailed information on the pre-identified key stakeholders who were going to be addressed or contacted during the course of the KT project. At the ex ante stage, this list did not need to be exhaustive nor complete but rather forced an early definition of each project's key stakeholders and target groups. Moreover, scientists were prompted to identify the stakeholders' sphere of action, function and how the stakeholder would be approached over the course of the KT project. All responses revealed a high degree of uncertainty in the ex ante stage, which points to the limited degree of experience of the participating scientists, who mainly originated from the natural science domain. In addition, it indicates limited knowledge on the question of who are the key nonacademic stakeholders for a given research topic.
During the in itinere interviews this early finding was somewhat reinforced. For instance, it surfaced that the development of tailored database portals required more time and capacities than originally anticipated due to the complexity of such products and the demands raised by nonacademic stakeholders. Dialogue approaches, such as workshops with a clear topical focus, were generally perceived to generate more positive effects by the researchers in contrast to online-surveys, in which the limited number of responses were perceived as critical and somewhat frustrating. By and large, all project participants perceived stakeholder communication as a challenge, often highly dependent on trust between the researchers and respective stakeholders. The different expectations and time scales of actions and the issue of continuity of stakeholder communication surfaced as additional challenge to some projects.
Lastly, in the ex post assessment researchers were asked to identify their set of key stakeholders who have been involved in, shown interest in, or have been affected by their project. In comparison to the ex ante assessment, researchers were much more readily able to clearly define their stakeholders and key actors, their respective specific demands and interests and the methods on how collaboration and exchange with specific actors took place.

Reflection on accompanying knowledge transfer process assessment
At the ex post stage, a feedback opportunity on all aspects of the KT process acted as opportunity for self-reflection on lessons learned during the individual project life cycle. Furthermore, to share, reflect and validate the findings across all KT projects, a three-day workshop of all KT project members was conducted in late 2016.
By and large, all interviewees felt very positive about the accompanying KT process assessment framework, as it helped to navigate explorative new KT paths. Moreover, the deliberations and feedbacks obtained helped the project management to address administrative challenges to facilitate KT further.
Several respondents stated that they had positively learned about the potential challenges, benefits and pitfalls of KT processes, and what types of methods and strategies resulted in prolonged, constructive and sustained science-stakeholder dialogues. For example, modelling approaches often did not match well with stakeholder expectations and time horizons, data products were often perceived as useful but, more often than not, regular maintenance duties took up more time and resources than expected. Dialogue formats were found to be helpful to gain better understandings on stakeholder interests and central underlying research questions on specific topics, but preparation time for invitation to workshops was often underestimated (Krause 2018). The self-reflection and learning options were underlined by several comments of the partaking natural science scientists such as: The initial planning of the (data products) was too naïve. The implementation effort was much higher than initially expected.
Scientist, data product development [T]he learning curve on knowledge transfer and related processes was quite steep and much was done in a trial-and-error way.
Scientist, dialogue format project stakeholder workshops with representatives from the local fisheries sector, aquaculture, fish processing industry, regulators, NGOs and research, in which the impacts of climate change on the German North Sea environment and their implications and potential transformative measures for adaptation to these changes for policies and society were discussed establishing an online portal for marine litter (LITTERBASE, see http://litterbase.awi.de) which summarizes scientific results in global maps and figures for policy makers, public authorities, media, scientists and the general public development of a statistical forecast system for German waterways based on a multiple linear regression model, which has been used to forecast extreme events and to provide early warnings of predicted floods or low flow situations for the most important German rivers (e. g., Rhine, Elbe and Danube) (Ionita et al. 2015) > RESEARCH

Scientist, dialogue format project
The KT project provided the possibility to approach a new topic in an explorative way, not knowing what would be the outcome of the project. This was a unique chance to develop knowledge on knowledge transfer processes.

Starting from small projects/ideas -big things can develop.
Scientist, modelling and forecasting project Several project members stated that they felt much more confident in identifying key stakeholders in future research, and what opportunities and prerequisites are needed for sustainable KT engagement.
[I]f the stakeholder communication process shall be a steady and fruitful one, it should be profitable to both sides. Participants (at stakeholder workshops) expressed more willingness to participate actively in a research project if they can help shape the research agenda and core research questions from the very beginning.

Scientist, dialogue format project
Due to this project we were able to contact additional stakeholders.
Scientist, data product development "Trust" -it is imperative that the individual who makes contact with the stakeholders remains the "contact person".

Scientist, modelling and forecasting project
The project-accompanying KT process assessment was perceived as a supportive guiding tool throughout the projects' lifetime. However, several respondents encouraged the develop ment of more direct and regular stakeholder feedback loops with in the process assessment in order to get more balanced and timely insights into the quality of KT activities. These statements point to the central role of creating safe learning arenas that allow sci en tists to move out of their conventional fundamental (not only natural) research spheres towards more integrated research op por tu nities. Constant feedback loops help to foster multi-loop trans for mative learning between KT projects as well as within the accom panying research project (Van Breda and Swilling 2018). As indi cated also in Weith et al. (2019, in this issue), such accompa nying research processes supported the dissemination of existing knowledge and identified successful ways of implementing knowledge to different realms of producers and users of knowledge. The findings indicate further that only if (fundamental) sci entists are acknowledged and endorsed by their respective insti tution for their efforts towards KT, scope emerges for potential transdisciplinary research forms.

Principles for good knowledge transfer processes
Our results reveal that our point-of-departure -successful KT at the science-stakeholder interface requires good processes -seems to be instrumental. So, what is a good KT process? Our findings indicate that successful KT has its roots in the type of impulses initiated by KT processes, next to how and in what ways have they been coupled internally and externally.

290
We applied the grounded theory approach (Glaser and Strauss 1967) to the empirical findings by clustering and comparing the central findings along the different stages of the formative analysis with the KT results of each project. Thus, we were able to identify which process properties were linked to satisfying or dissatisfying results. Despite the wide topical range of the investigated projects, a set of principles and related criteria for successful KT projects could be identified, which supports the assessment and evaluation of KT processes at the AWI as a showcase for a fundamental natural science institute (table 3). Such guiding principles as have emerged from the accompanying and reflexive research pro cess are meant to steer future KT processes (cf. similar reflections in the field of transdisciplinary research by van Breda and Swilling 2018).
Depending on the contextual nature of each respective KT project, the internal weighting in the sense of the applicability of the different criteria of each principle can vary to a certain degree. This can be attributed to the specific priorities, goal-orientation and scope of each KT project. For example, projects with strong focus on dialogue approaches are different from projects that develop an information database or modelling tools. Subsequently, we discuss the principles and some of the related criteria derived from the findings of our KT process assessment mirrored with international research.
Have an appropriate process set-up Based on our findings, it seems instrumental to have a sound setup and embeddedness on multiple levels for the respective project. This ranges from the available skill-set and individual experience of the respective researcher involved in the KT project, to the ongoing support within a core research group, to practicing equitable information flow in the process between all participants, in and outside academia (Muthusamy and White 2005). All of which link and frame the formal and informal processes of joint or mutual learning between actors from diverse backgrounds in terms of "social learning" (Singer-Brodowski et al. 2018). Thus, despite KT projects appearing to consist of basically the same steps as monodisciplinary research (problem exploration, hypothesis formation, research design, data collection, analysis, reporting, evalu ation), there are substantial differences. These differences are primarily in the degree of emphasis on the different phases, the number and type of actors involved, the diversity of their backgrounds, and the type of activities that take place in the different phases (Spaapen and van Drooge 2011). In this sense, KT processes can (and must) acquire significance within the context in which they are placed (Wenger 1998).
As all of these activities are rooted in an institutional environment, they have to be embedded within an overall strategic longterm commitment to KT made by the institute (Keramati and Aza deh 2007). Only when the required resources (e. g., personnel, infrastructure, time, funding, etc.) and the(institutional) environment, in which the process operates (i. e., laws, regulations, policies, research constraints, etc.) are appropriately considered and all partakers of this process identified and incorporated, positive scope for successful KT can be expected.

Deliver a meaningful output
Research insights need to be tailored to specific needs of the respec tive audiences in order to develop relevant or meaningful outputs. What constitutes relevance or meaningfulness is part of an ongoing negotiation process between academia and society and may vary widely for different social groups and contexts, and different scientific disciplines alike (Hornidge 2014). For contextualization of research findings via KT with stakeholders, the require -  possess the required skillset to plan, implement and dynamically adapt the process align with long-term research strategy of the Institute and embedded in core research expertise are embedded in a strategic and sustained long-term commitment to KT enact equitable information flow between all participants of the process show consistency of communication and clear accountability contain effective feedback loops (internal and external) that promote continuous validation and learning are tailored to a specific need with clear problem focus are translated to recipient requirements are driven by demand promise continuity are based on scientific integrity (based on sound science) stimulate interdisciplinary and intra-institutional cooperation provide sound and comprehensible advice and innovation improve institutional development and capacity building foster trust and exchange across communities create and maintain know-how in science and society support informed decision-making strengthen creditability and visibility ("honest brokership") enhance transparency and accessibility ments of actors from scientific and societal realms need to be understood in order to design a targeted output (Regeer and Bunders 2003). However, it is difficult to capture "relevance". Indeed, as pointed out by Weith et al. (2019, in this issue), it is necessary to be specific from the onset whilst developing the transfer goals: what should actually be transferred to whom, when and why? By focussing on the processes, this ongoing negotiation can be traced and becomes visible (Spaapen and van Drooge 2011).

RESEARCH
Hence, a meaningful output requires a clear problem focus and must address voiced demands on the side of the researcher and the recipient.Transparent discourses on priorities and expectations, whilst retaining scientific credibility and transparency, foster trust between all actors within a KT process. Matuleviciene and Stravinskiene (2015) identified two core factors responsible for creating trust in stakeholders: corporate reputation and organizational trustworthiness. Pursuing a meaningful output can thus justify and defend innovative thinking and risk-taking in a research community. Scope for flexibility in the creation and assessment of outputs is needed that relaxes the current tight coupling to traditional research metrics (Kates et al. 2001). The assessment of such KT outputs in order to answer the call for evidence of impact (BEIS 2016) hinges on novel types of compilations of these.

Align with institutional values
Being part of an institution, in our case a research institution, KT has to reflect and embrace the institutional values (Hornidge 2014). Turner (1997, p. 6) defines a social institution as "a complex of positions, roles, norms and values lodged in particular types of social structures". Science can be viewed as one type of such social structure, in which values and norms arise from within but are also influenced by the larger national and international institutions in which they are embedded. This is reflected, for instance, in the general agreed consensus to safeguard good scientific practice by self-regulation in science (DFG 2013), or the increasing adoption of the FAIR Guiding Principles (Findable, Accessible, Interoperable, and Re-usable) for scientific data management and stewardship (Wilkinson et al. 2016).
The heightened recognition of the institutional demand for increased transparency and democratisation of knowledge has brought forth the above-mentioned multitude of KT projects in the AWI during the last decade. The projects were all exploratory and society-focused in nature, while still focussing on the institute's core research competencies. Projects that did not align to a certain degree with the inherent institutional values received little support and recognition within the institute. This in turn di-rectly counteracts attempts at creating sustained KT processes from within. Correspondingly, Davis (2009) emphasizes the need for relevance of any good process to the underlying institution or business.
Projects which focussed on the institutional values resulted in sustained KT processes with long-term perspectives whilst providing scope for new interdisciplinary and intra-institutional cooperation and joint learning. These successful activities, in turn, help to develop KT-friendly research norms and expectations at a disciplinary and institute level (BEIS 2016).

Create added value for the science institution and society as a whole
The decision on which science ventures should be supported by the research institution and why, is regularly subject to negotiation with the priorities of society (Markus et al. 2017). As the complexity and volume of KT processes and outputs are continuing to grow, an ever-growing burden is placed on the research institution to maintain and improve these products , Hampton et al. 2013. This issue became especially clear after analysing results from projects with a data-product-driven focus. Data products, such as web portals and databases, are tools designed to make scientific data and results better accessible and present singular data points in a bigger picture to a wide range of target stakeholders. This service, however, comes at the price of maintaining personnel with the necessary expertise and infrastructure during and after the project period, in order to sustain any positive effect. Hence, the added-value to the institute and society as a whole must offset these increased efforts. The same princi ple holds true while assessing projects focussed on sustained stakeholder dialogues (Krause 2018). A key example for the importance of such sustained dialogues is the German Arctic Dialogue (Rachold 2018). The latter has generated an added value towards informed decision-making, while also shaping emerging research topics and forging new priorities. However, such strategic alliances need competent and engaged individuals -on the science as well as the societal side (Muthusamy and White 2005). The alliances can eventually lead or contribute to "knowledge infrastructures", which Edwards (2010, p. 17) defines as "robust networks of people, artefacts, and institutions that generate, share, and maintain specific knowledge about the human and natural worlds". These infrastructures produce Mode-2 knowledge that cannot be authoritatively encoded in traditional forms of scholarly publication alone (Nowotny et al. 2003). Rather, it must be socially distrib -> Accompanying formative and summative evaluation fosters reflexive turns on the quality of transdisciplinary research projects by guiding the adaptation of tools and methods that improve immediate transfer process and outcomes, whilst providing scope for learning at project and institutional level.
uted, application-oriented, transdisciplinary, and subject to multiple accountabilities. Indeed, all assessed projects, regardless of their topical focus, were rooted in both credibility and visibility, as well as transparency and accessibility of the science institution. These act at the same time as prerequisites, outcomes and added value of good KT processes.

Central conclusions for research accompanying knowledge transfer processes
The above described principles have proven to be prerequisites for establishing successful KT processes within the scope of the analysed projects. These principles were identified through an accompanying research effort that aimed to capture central KT processes from start to "finish". Reflecting on the role and potentials of such accompanying research in KT, we conclude: Assessing the underlying processes of KT continuously allows for direct influencing of processes and outcomes. We need to focus on the processes underlying and preceding the impact of our research. Accompanying KT projects and their related processes from the onset fosters a better understanding of how specific science-stakeholder interactions are shaped and driven, thus enabling to effect changes and optimize eventual outcomes. Highlighting expectations, identifying potential indicators and approaches as well as methods to optimize these can all serve to improve the overall quality of interactions. However, the best-tailored KT project may be subject to "failure" if factors outside the direct sphere of influence of the project members override the processes. The identification of these externalities may release the "burden" on researchers to prove the practical value of their science to society.

Multi-level feedback loops promote capacity building and institutional learning.
Thanks to the continuous bi-directional feedback about the KT processes in each of the projects, all actors (scientists, project support, financial administration and outside stakeholders) were aware what issues, problems and opportunities each project faced. These reflexive turns support the adaptation of tools and methods that improve immediate transfer processes and outcomes, whilst providing scope for learning at the project level. Such insights help to clarify the purpose, expectation and form of the interactions between science and society and to possibly adapt institutional priorities and research topics. These internal feedback loops on the institutional level may support KT processes by creating supportive administrative structures as well as a culture of recognition for such activities.

Institutional self-reflection improves quality and traceability of KT processes and their contributions to solving societal problems.
The framing of the interaction processes in an assessment supports the quality and scientific rigour of KT, whilst emphasizing the relevance and workability of links between science and society. It supports internal reflections on the role of science contributions to society. Having a standardized assessment framework for these processes in place makes the efforts more visible, thus promoting a "culture" of KT inside an institution. Knowledge about processes is knowledge about how and when KT works, and when it does not. This sidesteps the risk to jump to wrong assumptions on what science-stakeholder interactions can potentially achieve.

KT needs to be rooted and endorsed in different structural, organizational and cultural levels of the institutions involved.
KT requires multi-level institutional impulses, instruments and feedback loops in order to provide a safe navigation space for institutional learning at the science-stakeholder nexus. Only if (fundamental) scientists are acknowledged by their institution for their efforts towards KT, scope emerges for transdisciplinary research forms. It is a continuous process which needs to grow from within the institution.

A prioritization of institutional norms and values and their related topical preferences is needed to operationalize the KT process principles.
What is captured and evaluated under the umbrella of KT may shift depending on the institutional norms and values, reflected by voiced interests on potential outcomes. Our KT process principles are not one size fits all but rather provide guiding rails to operationalize KT assessments. They could support for example evaluation procedures by addressing the questions of how to capture and judge science-stakeholder interactions and how to better link research findings to societal problem solving in the future.n Society and science are interlinked sub-systems and must be considered jointly, as both have specific priorities that are transferred in one way or other. This systems perspective requires a re-shaping of KT processes that take place within and outside of science, leading to a specific set of processes that result in specific outputs and impacts across the different societal realms and different time periods. To date, existing KT metrics focus on what is measurable -often at the expense of what is important for society.
A deeper view into the definition of societal relevance is needed. Bearing in mind that we have various forms of knowledge (in and outside academia) that all have their legitimacy, this points to the sovereignty dilemma on who defines what a good societal impact is (and for whom) -and how this impact can be accounted for. Furthermore, societal perceptions may be voiced directly or indirectly and may become relevant at different points in time, thus making it difficult to evaluate such somewhat intangible pro cesses.
We conclude that better insights into the impact of research may not start with the result, but rather it must start by asking how the research is conducted and for whom. There is not "right or wrong" but rather "better and worse". Societal processes generate new challenges for research, in which it is not only the question of how to handle scientific evidence-based knowledge but how to actively promote that knowledge by enabling good processes. This is timely as fundamental research institutions are increasingly being asked to provide context for the application of their