Evidence-based research impact praxis: Integrating scholarship and practice to ensure research benefits society

Effective research impact development is essential to address global challenges. This commentary highlights key issues facing research impact development as a nascent professional field of practice. We argue that those working on research impact should take a strategic, ‘evidence-based’ approach to maximize potential research benefits and minimize potential harms. We identify key features of evidence-based good practice in the context of research impact work. This includes integrating relevant research and theory into professional decision-making, drawing on a diversity of academic disciplines offering pertinent insights. Such an integration of scholarship and practice will improve the capacity of research impact work to make a positive difference for society. Moving the focus of research impact work to earlier stages in the research and innovation process through stakeholder engagement and anticipatory research can also boost its effectiveness. The research impact evidence base should be combined with the right kind of professional capacities and practical experience to enhance positive impact. Such capacities need to be developed through relevant education and training, for example, in participatory methods and social inclusion. Such training for research impact work needs to forge strong links between research impact scholarship and practice. Finally, there is a need for improvements in the evidence base for research impact to make it more practically useful.


Introduction
Leveraging knowledge to make the world a better place is a noble goal in research.However, it can also be challenging to develop the most appropriate strategies for non-academic impact objectives, intended beneficiaries and specific economic, social and cultural conditions.Faced with numerous challenges, we argue in this commentary that the emerging field of professional practice around the generation of research impact would benefit from more evidence-based approaches, where appropriate scholarship and professional practice are brought together into a coherent praxis.An evolution towards evidence-based research impact praxis is essential for progress in how research impact professionals operate.This essay sets out how research impact professionals and scholars can work together to develop improved strategies and practices.We argue that better, more socially responsible development and application of the best available scholarship will deliver more beneficial processes and outcomes both for society and for the research enterprise.
Research impact as a recognizable field of professional practice is relatively new, overlapping with other more longstanding research-related practice domains such as science communication and sustainability.This newer field has been developing differently around the world.Job titles such as 'impact officer' have become commonplace in some countries, such as the UK, in the last decade.In this essay, we use the term 'research impact professionals' to refer to those involved in managing or developing research impact, both as their primary employment (for example, impact officers) and as researchers who are also working to generate impact beyond the academy as a secondary aspect of their work.Non-academic impact aims to extend beyond academics and students to include industry, policy makers and different types of publics such as young people, migrants, or orthodox religious groups.This is an aspect of research and innovation systems that has recently come to the fore, riding the wave of top-down initiatives to promote non-academic impact, initially as a condition of research funding and, more recently, integrating into national research assessments.Example initiatives in different countries include the UK's Research Excellence Framework (REF), Italy's Research Quality Evaluation, the Spanish National Commission on the Evaluation of Research Performance, Hong Kong's Research Assessment Exercise and Australia's Engagement and Impact Assessment (Reed et al., 2021).However, there is growing evidence that top-down initiatives to promote impact beyond the academy may introduce unintended negative outcomes within the research and innovation systems.When coupled with limited research impact capacity or expertise within the institutions charged with delivering benefits to society, prominent negative outcomes may emerge.Examples of such unintended outcomes may include forms of de facto corruption such as elite capture (de Hoop et al., 2016) and conflicts of interest (Chubb & Reed, 2018; Watermeyer, 2019).The European concept of "RRI" (Responsible Research and Innovation) has similarly been criticized for being more of a policy prescription than a paradigm that is also supported widely and implemented from the bottom up (Gerber et al., 2020).Additionally, forms of cultural imperialism may extend the prevalence of social inequities and opportunity costs for the wider populations who engage but receive little or no benefit (Cooke & Kothari, 2001; de Vente et al., 2016; Watermeyer, 2019) from the research and innovation system.
To ensure research impact scholarship provides more relevant insights for the community of practice in this domain,

Amendments from Version 1
Reviewer 1 said, 'I encourage a broader survey of the literature.'Given that this is a commentary grounded in the authors' prior work, we don't feel this is required.However, we have added the quotation from the reviewer's article that was helpfully suggested in the review.Reviewer 2's comments have in part been integrated into the paper itself.This is because many of the points raise were astute but additional to the points currently made in the article.Nevertheless, this point has been added, 'The article would benefit from acknowledging the challenge of language around research impacts work, which has myriad other terms (e.g., public impact research, public engagement, public scholarship, engagement scholarship, etc.) and adjacent or intersecting fields (e.g., science communication, public policy, etc.)'.These reviewer comments are great, but we feel would be additional content that could go into a future commentary rather than being essential to integrate with the present manuscript: 'The commentary would benefit from greater clarity about proposed mechanisms to evolve the funding ecosystem, especially in the context of limitations in existing funding mechanisms and allowable expenditures to support engagement and research impacts work (e.g., Can research funds be applied to paying for the effort of professional support staff?Better support the development & scaling of participatory processes?),weighed against real limits in revenue sources & priority in resource allocation.' We are very grateful for the valuable engagement from both reviewers, who are doing excellent work in this space themselves.

Any further responses from the reviewers can be found at the end of the article
there is an ongoing need to engage with intended beneficiaries and identify the impact goals or outcomes that will drive the evaluation (Jensen, 2015b).Reed et al. (2021:3), in their characterization of research impact as "demonstrable and/or perceptible benefits…", emphasize the subjectivity of benefits that may arise from research, particularly when the benefits or advantages to one group may be perceived as damaging or harmful to the interests of another group (or the same group in a different time or context).Furthermore, situations may arise where the interests of beneficiary groups are poorly represented, and over-managed participatory processes may lead to forced agreements that simply maintain existing power structures (Cooke & Kothari, 2001).
The challenge here is partly structural, based on funding and incentive structures and the types of research that tend to be prioritized.Indeed, few research funding schemes incentivize genuinely co-productive research with diverse stakeholders.Limited co-production between researchers and practitioners can be a major barrier to impact, leading to the generation of unintended consequences for interests of groups who were not engaged or were not able to engage effectively in the process (Adams, 2008;Cooke & Kothari, 2001).For example, Fritsch & Newig (2012) conducted a case-survey meta-analysis of environmental management publications involving stakeholder participation (many of which were initiated and written up by researchers) and found a bias in outcomes towards the interests of over-represented groups, typically at the expense of alreadymarginalized groups.Such outcomes can damage the trust between research institutions and marginalized groups, with long-term negative consequences for both sides.Instead, researchers and practitioners need to consider the 4inimizies-competing interests of different groups as they change over time in response to changing social and political contexts or changes in the personal circumstances of participants (Sarkki et al., 2014), to avoid over-representing those most easily accessible to researchers (Colvin et al., 2016) and represent the diversity of perspectives and realities voiced by different groups (Moon et al., 2019).
Existing scholarship on participatory processes argues for the importance of giving power back to intended beneficiaries in an attempt to reshape the desired outcomes and terms of an evaluation based on their perceptions (e.g., Estrella & Gaventa, 1998;Guijt et al., 1998;Villaseñor et al., 2020).Drawing on research and evaluation methods from the social sciences and the arts and humanities, the participatory process emphasizes the value of inclusion by ensuring a plurality of voices and perspectives.Specifically, the process of making research relevant for intended beneficiaries should give marginalized voices weight when deciding what beneficial outcomes count as research impact (Coemans et al., 2015;Reed et al., 2021;van der Vaart et al., 2018).
As the policy drive for 'impact' spreads and gains strength (Edler et al., 2012; Oancea, 2019), the cadre of research impact professionals employed to support the development of nonacademic impact continues to expand, alongside the quantity of researcher time and resources dedicated to impact work as a secondary or tertiary activity (Jensen, 2020b; Jensen & Holliman, 2016; Wróblewska, 2021).However, this expansion of research impact staff is not yet accompanied by formal training, such as the development of master's degree programmes or widely recognized certification schemes that could underpin a shared understanding of professional practice in this domain.This paucity of formal training raises concerns about where these research impact officers are supposed to look for insight and guidance about effective professional practice.
A loose collection of academics, consultants and representatives from government and non-governmental agencies, funding organisations and private companies offer research impact advice while drawing on a diverse range of backgrounds and expertise.The need remains to clarify and develop a coherent framework, shared understanding and working consensus among these actors of professional practices to most effectively identify, evaluate and evidence impact within the research and innovation system, while minimizing the potential for unintended outcomes and risks of harm and improving societal benefits.
Here, we argue that this need for professionalization in research impact work is best addressed through an integrated approach with support in scholarship and practical research experience.

Key challenges
With the growing expectations for research impact work, both professionals and scholars in this domain must further develop their capabilities for critical self-reflection, evidence-based practice, and robust impact evaluation.Indeed, the development of these capabilities will underpin necessary long-term progress for research impact policy, practice and scholarship.Across our diverse fields, we have been involved in research impact practice and scholarship as these domains evolved over the years.In our work at the interfaces between policy, practice and scholarship, we have helped address numerous and varied challenges that we, and many other scholars and practitioners, encounter relating to research impact processes.We have previously highlighted a range of these challenges (for exam-

Making impact scholarship relevant
First, there is the challenge of making research impact scholarship relevant to professional practice and intended beneficiaries.Few academic publications on the theme of research impact attempt to establish why this work matters for professional practice, nor explain how to address results and findings.In addition, the diverse terminology that is used to refer to impact literature and professionals can pose a barrier to identifying relevant work that has already been published.Moreover, findings, insights or implications presented in academic publications may remain inaccessible to practitioners when obscured by disciplinary academic jargon and opaque writing.This challenge was highlighted by Bayley and Phipps (2019, p. 4) as a defining aspect of 'advanced' levels of 'impact literacy'.
Critically engaged with the evidence, understands there is a body of expertise, knowledge and tools which can underpin practice and is able to (i) synthesize, (ii) critique and (iii) add to/extend it.
Faced with such communication challenges, participatory and evaluation processes can be practical and valuable (Boydell et al., 2012) to help clarify the relevance of research topics that are sensitive or hard-to-verbalize.These processes can also support dialogue with those the research intends to benefit, thereby facilitating more fruitful knowledge exchange.However, maintaining such dialogues efficiently, for example, by using evaluation surveys, can be challenging.Evaluations require that feedback is listened to, acted on, and changes are made to address inadequacies.Participatory approaches often work with small numbers of people, but it may be essential to broaden the number of informants to clarify the relative prevalence of key issues that impact the welfare of a larger population.Extending the evaluation of success indicators from small to large samples requires complementary tools and resources for more quantitative measures of change over time (e.g., see Morgan, 2007;Morgan, 2014).Inspired by Heneghan et al. (2017), we emphasize the need for research impact scholarship to provide relevant, accurate and timely insights that practitioners can implement.

Making impact scholarship accessible and applicable
Once the relevance of research impact scholarship is clarified, it is necessary to ensure findings and insights are accessible to practitioners.Although open data and open methods are more common with recent advances across research and innovation systems (e.g., Piwowar et al., 2018), many research findings are primarily published in English and behind paywalls.Furthermore, efforts to translate findings from impact scholarship into non-English languages are currently limited and ad hoc, leaving significant scholarly contributions inaccessible to many global researchers and practitioners in non-English speaking and low-income countries.A prerequisite for this would be the applicability of theoretical frameworks and assessment methods in the communities of practice.Applicable or applied research can lead to direct actions or solutions that address the specific needs of intended research beneficiaries in a range of contexts.In contrast, basic research may help to define or describe problems but may seem more abstract or conceptual, appear less relevant to practice and be more challenging to demonstrate how it can be applied.Indeed, the majority of published research describes or defines problems (known as "mode 1"), often proposing and testing hypotheses aimed at leading to a generalisable theory that is broadly applicable across many different contexts.The main alternative is research that focuses on the more localised experiences of individuals or groups in specific contexts or situations (known as "mode 2") in which knowledge is generated.Recommendations based on narrow individual case studies may overgeneralize beyond the context in which the study was originally conducted.Scholarship should work on clarifying the conditions for transferability of research findings, rather than assuming universal applicability.Collaboration between scholars and practitioners will help to improve the relevance and applicability of research impact findings.
Considering how much both the rationale and the assessment of non-academic impact are increasingly interwoven with the respective research systems and cultures, research funding processes greatly affect research impact work.Formal review procedures organized by funding bodies and even regulatory initiatives by science policymakers impinge on definitions of impact, as well as the perceived legitimacy of different research impact goals and approaches.Indeed, reviews of grant applications and funding decisions comprise key moments where policy meets practice, but the role of research impact scholarship in such concrete tasks is limited or non-existent in our experience.Academic methods and models used in research impact scholarship need to be made applicable in ways that allow a direct transfer into these kinds of specific, critically important tasks.Further changes to who reviews proposals and projects may be needed as those reviewers "need to be selected, briefed and possibly even trained with regard to their capability to assess different degrees of engagement and participation" (Gerber, 2018, p. 2).Practically speaking, the key applicable points from scholarship relevant to non-academic impact will need to be operationalized in the form of criteria for proposal evaluation, reviewer selection and reviewer briefings, guidance for grant-writers and systemic changes and infrastructure needed to make research impact more effective and inclusive.
Ensuring and increasing the quality of impact scholarship Research impact scholarship needs to avoid questionable practices that could produce errors and undermine accuracy (John et al., 2012) in findings and implications.There is a risk that research impact scholarship can have errors, mistakes, or inaccuracies that are subsequently applied in practice, leading to unintended or undesirable outcomes.There are, for instance, calls to extend research methods from other disciplines, such as medicine, that include standardizing sets of core measures or indicators that can be consistently reported and enable meta-analysis (Nichols et al., 2021; Stewart et al., in prep.).Despite some notable initiatives in specific impactrelevant domains (consider the work of Conservation Evidence in environmental science; Sutherland et al., 2015), there are very few systematic reviews of research impact scholarship.
Evidence synthesis, systematic reviews and meta-analysis that compare research designs, methods, results, and findings across studies can provide a more reliable basis for recommending changes in practice.Steps to increase research quality also help ensure that methods and results become more comparable across studies and are generalized correctly.This will improve the probability that efforts to make such scholarship more relevant and applicable will not be in vain.In contrast, a lack of comparability hampers generalizability and thus again the applicability of the scholarship at large.Many evaluations of research impact lack key methodological details and do not even meet the quality criteria for being included in systematic reviews.Most commonly, researchers trained in the natural or physical sciences attempt to use social science methods to evaluate impact, including surveys, interviews and focus groups.However, there are often shortcomings in the research design, methods, and analysis that limit rigour from these studies.For example, quantitative evaluations may lack adequate sample sizes for statistical power, replication, baselines, or control comparison.Whereas qualitative evaluations may lack sufficient triangulation between sources or critical interpretation of findings (e.g., Jensen & Laurie, 2016; Jensen, 2020b).

Karcher et al. (under review
) have recently completed one of only a few systematic reviews of impact evaluations.Findings from this review have shown that evaluations often referred to the evidence of research impact as outcomes that create products (e.g., reports, maps, tools), enhance the usability of knowledge (e.g., credibility, salience, legitimacy) or improve social connection (e.g., networking, awareness, learning, trustbuilding) between stakeholders.While the objectives of evaluated interventions often aimed to achieve policy, economic and societal impacts, evaluations rarely collected evidence on these outcomes.These results may represent a failure of research to generate impact or reflect shortcomings in impact evaluations in the available published literature, including methodological limitations (see Jensen, 2020c) and misalignment between the evaluation timescales over which impacts occur (Gow & Redwood, 2020; Morris et al., 2011).

Evidence-based pathways toward research impact
In our view, there are several concrete measures that science policy and research funders, universities and even stakeholders can take to foster a more evidence-based and thus effective research impact: • The praxis of applying scholarship on how to anticipate and influence non-academic impact of research and innovation systematically, will foster more effective and resource-efficient impact actions.Growing policy expectations for research impact will most likely drive further professionalization to design and implement more evidence-based approaches.This will make both the impact activities and their evaluation methodologically more robust.
• For practitioners to explore and potentially apply the existing body of evidence, the relevant scholarship needs to be conveniently accessible.Since the scholarly publishing system is unlikely to make the required changes to deliver this voluntarily, it will be imperative that research funding organisations further incentivize not only open access but also open data and open methodology.This would foster the comparability and generalizability of available evidence and encourage systematic reviews that sift through the body of research and provide professionals with more quality-assured evidence they can use.
• It is also a prerequisite for this praxis to be addressed from both sides: by impact professionals being reflective and open to potential changes in established practices, and by impact scholars ensuring direct applicability of their findings.The reflexivity in practice must include honest self-assessments of the limitations of one's work so that evidence that potentially invalidates previous practices will not be rejected outright.Ideally such praxis would evolve from mutual learning through collaborative action research, and by sharing experiences that may benefit the wider community of practice.
• All pathways toward more evidence-based research impact will also need to increase awareness among funders, research performing organisations and individual researchers of the need to anticipate and act upon the potential benefits and harms of the research and innovation they are pursuing.Taking well-designed steps to involve stakeholder perspectives early in the research and innovation process can pay dividends in long-term impact.This is where research stakeholders in general, and the potential end users of technological and social innovations in particular, must be empowered to voice their needs and expectations in a way that can have a real influence.
• In response to this increased awareness, impact professionals will require significantly more capacity building than is even offered now.Whether integrated into the syllabi of existing professional development, master's or PhD programmes, or offered as stand-alone programmes, it should go without saying that more evidence-based approaches can only be fostered by similarly evidence-based teaching and training, which is not the case yet.
• One of the key aspects of such training must be to highlight the importance of actively including marginalized groups in the design and implementation of research impact actions using evidence-based communication and involvement strategies (see Jensen, 2013; Kennedy et al., 2018).Existing scholarship on social inclusion from academic fields such as sociology offers a wealth of untapped insight that can make practice more effective.
• In addition to professionalizing research impact work as such, the activities in this field will also need to be monitored and evaluated more systematically, and ideally also more comparatively (Jensen, 2014; Jensen, 2015a).This assessment needs to be of sufficient methodological rigour (e.g., Jensen & Laurie, 2016; Kennedy et al., 2021), and ensure that appropriate ethical principles are considered, such as informed consent for participation and responsible data protection and management (Jensen, 2020a).Maintaining transparency and openness regarding the nature of funding and its organizations and institutions, can have a positive influence on the design of research impact activities (see Gerber, 2014).
• Ensure resource-efficiency to maximize opportunities for positive research impact activities.
We recognize that the suggested pathways forward will always be affected by the perspectives of researchers, practitioners and intended beneficiaries of research impact activities.These perspectives will also be influenced by institutional, local, and cultural circumstances.

Conclusions
In this commentary, we have expressed concern that research impact professionals may not sufficiently benefit from relevant, accessible, applicable, and quality scholarship.We contend that this is an ideal time to consider the trajectories of research impact before problematic professional norms in this still-forming field become too ingrained.However, the domain of research impact is not yet an established 'field' in the conventional sense.Indeed, it is still a loosely developed community of practice that comprises researchers, various research support officers and other professionals and staff at research funding organizations.While this status for the domain of research impact may make our call for evidence-based practice seem premature, we believe this is a meaningful discussion that must involve research impact stakeholders and those who work across research-practice boundaries.
This commentary aims to nurture reflectiveness in this community of practice by starting a conversation about the effectiveness of impact-related practice, evaluation and scholarship.In our view, much 'evidence' in the domain of research impact will need to be challenged and considered provisional for quite some time.This provisional nature of findings will often happen at the frontline of evolving areas of scholarship.In the meantime, there are many well-established and well-evidenced insights and theories that can be safely used to underpin evidence-based research impact work.While RFOs today may be satisfied with a general increase of 'any' non-academic impact, regardless of why and how those materialize, their expectations are certainly growing.At the same time, other societal stakeholders are increasingly demanding their voice be heard further upstream in the research and innovation process.In response to the combination of these top-down and bottom-up trends, capacity building initiatives are slowly starting to respond.Research and impact professionals will increasingly be trained in assuming a more anticipatory approach to analyse and account for the risks and benefits of their research and innovation.This can empower them to plan and implement more appropriate impacts, for instance avoiding detrimental implications and bolstering positive outcomes for marginalized groups.
More than just ascertaining and appreciating non-academic impacts, RFOs can (and must) increasingly demand a more formative and thus evidence-based approach.Impact officers could, for instance, learn to professionally manage focus groups with stakeholders already during the research design phase.For the time being, most policy frameworks, such as the Research Excellence Framework in the UK insufficiently incentivize such upstream models by requiring that impact evidence be linked to specific research outputs (most often, journal articles or books).Such incentive structures have to be very carefully calibrated to ensure that they are rewarding long-term pro-social outcomes.
We argue that collaboration between impact scholarship and practice will improve the relevance and transferability of research impact.Furthermore, improved quality in the available evidence will offer the most significant practical foundation for improving research impact work, now and in the future.We are inspired by Heneghan et al. (2017) when we emphasize that the direction for research impact scholarship should seek to provide relevant, accurate and timely insights that practitioners can implement.
We intend to ignite further discussions about the principles and practices of evidence-based research impact.We also want to address the everyday challenges and experiences of using current evidence to inform and expand research impact development.This is the only way that evidence-based research impact can live up to its potential in a world where it is increasingly needed.

Key messages
1. Professionals working in the emerging field of 'Research Impact' can make their activities more effective by applying relevant evidence from the social sciences to improve their strategies and methods.As prerequisites, relevant scholarship on research impact must be identified and made accessible.Scholars in this domain should extend efforts to make their work directly applicable to practice.process.It will also be imperative for RFOs to train and guide research applicants, reviewers and evaluators in how to meet their impact expectations.In general, all impact activities need to be evaluated more systematically with sufficient methodological rigour and consideration of ethical principles.
This article addresses the gap between academia and practice and the professional development of individuals who span both domains.I felt a little unclear as to who is the intended audience for this paper.Is it about, and targeted at, professional staff working in academia in roles similar to myself, or practitioners that have become involved in academia, perhaps as Professors of Practice or of Research Impact, or external practitioners in organisations and institutions that engage with the outputs of academic research?Overall, I think there are some edits that could be made to this paper to give it more clarity for reader.Having said that it did make me think about professionalization of the roles of people working in support of research impact and the training that would benefit them.Also, how do you recruit, induct and develop people in the type of adjunct roles that span both academia and practice or policy?I would agree that there should be greater recognition of impact support roles which will benefit both academia and their external research impact beneficiaries.

Clarity around terms and focus:
The terminology used in discussions related to research impact has been raised by the first reviewers and the article now comments on the inconsistent terminology used in the domain.However, the paper could be more explicit and precise and this would greatly help the reader.For instance in the Plain language summary the phrase 'Those devoted to making a difference using research' is vague and I think that, overall, this summary could give a much clearer picture of what the paper is about.While some amendments have been made following the first two reviews, I think there is more that could be done that would give the paper greater impact with its intended audiences.I would recommend inserting a small table with some definitions of terms.I have a query about the term 'evidence-based research impact praxis' (referred to in the title and on page 3) and what it is that the authors are advocating here -are you trying to make impact from research more relevant or are you trying to fundamentally change how academics do research?In the Conclusions you refer to 'evidence-based practice' which is obviously not the same.
In the subsequent paragraph on page 3 -with the existing definitions, all research academics could be described as 'research impact professionals' but I assume that is not the intention.
Framework and principles: David comments that there is never likely to be a coherent framework widely accepted and practiced but there could be some principles that will translate across many contexts and recommends to think of principles rather than a framework.I think this is a very useful suggestion.As both reviewers have commented, there are considerable differences across disciplines both in what is academic scholarship and rigor and what is impact.So, some generic cross-disciplinary principles would be useful and have the benefit of broader applicability.At the moment I find that the paper is focused on the expertise of the authors (evaluation, participatory research) and while I appreciate that they want to promote these approaches, they could recognise that there are other perspectives/approaches that maybe also be valid.If the authors are seeking to provide some principles that can benefit academia widely then I think they should look more deeply at what works (and does not work) in different areas.I would add to this that the nature of impact across disciplines is vastly different and in this context I find it hard to believe that there is a generic approach to professional development around research impact that can be applied and recognised across all disciplines.Showing how your methods could work in other areas would have value, (see comment regarding examples below).

Specifics and real-world examples:
I agree with David that the commentary lacks real world examples of linkages between scholarship and practice.In advocating participatory research approaches, the paper could give some examples of how this approach has been used successfully in different disciplines, perhaps including STEM areas.Also, the point has been made that much impact comes out of discipline specific research and I think that there is not enough recognition in the paper that other approaches are valid and can have considerable impact, without necessarily being participatory or involving co-production.For instance, much funded research does require a significant element of collaboration and partnership with the intention of generating non-academic impact -this does not necessarily mean that the collaborators are actively involved in the implementation of the research.Currently I don't think that the paper gives sufficient recognition and tangible examples of how impact scholarship varies across specific disciplines and how practitioners can access and engage with those different contexts.
Practitioners also often seek expertise from interdisciplinary teams and while there might be the desire within institutions to work in this way there are still many barriers to effective interdisciplinary research that can solve real world problems.How can the development of impact professionals help address these challenges?
Training and professionalism: Elyse comments that different forms of expertise are needed at different stages in the process.The authors might clarify where in the system they believe the training and professionalization efforts are likely to have the greatest impact -e.g.more faculty with translation skills, expanding professors of practice, deepening commitment to professional staff to work with faculty.David points out that impact practitioners face the same barriers (as academics) -lack of time, lack of expertise, lack of incentive to engage with impact evidence and recommends to add to the paper courses available to impact practitioners.
I am fully supportive of the idea that research professionals (broadly defined) need greater recognition both within academic and externally.I support the idea that training and professionalization of research impact professionals will help with the academia-to-practice link.However, I struggle to see that a Masters route is the solution to this and can't see such an offer being appealing to people that I work with or practitioners that we engage with.I think if you are to make a case for a new Masters path there needs to be a stronger argument presented to back it up and clarity about who should participate is such a programme.But for me, of far more value, is the short CPD-type of professional training that is available and I am surprised that the paper has not mentioned organisations such as ARMA, UPEN, Praxis Auril and, for instance, the Canadian Association of Research Administrators, as well as international examples such as Society for Research Administrators International (SRAI), International Network of Research Management Societies (INORMS) etc. (I am sure there are many more international examples).These organisations do offer networking, conferences, briefings and training.No doubt there are gaps and more could be done that brings academics, impact professionals and practitioners together but I don't really see that gap being filled by a fairly narrowly defined Masters programme.
With regard to evaluation techniques and developing impact professional skills in this area, it is worth mentioning that for the most part, when we are looking for evidence of impact, that evaluation needs to be independent and undertaken by an external body, i.e. you can't be the evaluator of your own impact.

Minor corrections required:
The affiliations are slightly wrong, I think Mark Reed should be subscript number 1.

○
There is a typo in the Plain Language Summary: '2inimizin'.

○
There is a typo on page 4 '4inimizies-competing'.Reviewer Expertise: I am a specialist in research impact and engagement.My academic background is within the business and management discipline and my research focuses primarily on equality, diversity and inclusion.My research has covered retailing, police and construction sectors.I have also undertaken research on international franchising in retailing.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
2. Both the title and the contents of the authors' work allude to the value of co-creation (between academics and stakeholders/practitioners) in generating research impact.I fully support the idea.In this respect there are at least four pieces of research the authors may want to include/review in their paper: Halvorsrud et al. (2021 2 ) systematic review on the effectiveness of the co-creation of research, exploring which are the optimal mechanisms for successful co-creation, which outcomes are of relevance and how power is shared and knowledge generated.
○ van Dijk-de Vries et al. (2020 3 ) work on how a co-creative research approach could foster impact by increasing the researchers' understanding of the value of co-creation and the successful engagement of stakeholders in the research project.

○
Stier and Smit (2021 4 ) paper on how co-creation could increase the uptake of scientific knowledge.Some 11 conclusions are drawn including the allocation of adequate resources, a common agenda, reconciling differences in several domains, addressing questions of impact, validation and valorisation from the outset, and so on.3.In a number of places in their paper the authors refer to training/capacitation of academics as a successful strategy to enhance the impact of research, to that I agree.I do, however, support this should be part of the culture of the organisations these academics work for, not just training programs, but as part of the set of values, mission and vision (culture) of research organizations to be successful.
4. The authors mentioned on many occasions the importance of stakeholders.I believe their mapping, identification and involvement in the early phases of research is crucial to fostering impact thereafter.One of the first phases in this process would be the identification of the research needs such academics are willing to respond to.This is a major issue to maximise impact: addressing a real need with stakeholders' involvement in an early stage.
5. The points mentioned above indicate that impact needs to be addressed from the design of the research that is carried out.The authors and reviewers have reflected this idea in the paper, but I believe it is worth stressing that point: impact from design.
6.There seems to be much to be done on non-academic indicators of impact, that is, on which ways can we appraise impact.Further, it is surely not enough to agree on a given set of indicators but on how impact is really achieved.In this sense, there are many published practices on how qualitative Impact Narratives are being used to capture and enrich that, identifying barriers and facilitators, and connecting individual research cases to context variables.There is very little on this in the authors' paper, and much less on the difficulty of tracing impact, that is on how to lawfully attribute any plausible impact (on the social, economic, environmental or health domains) to the research carried out.
7. In their section on "Evidence-based pathways toward research impact" I would include: There is a growing need to incorporate research impact as part of the editorial mandates, maybe by means of new formats, methods or editorial priorities.

○
There is a need for Communication/Reporting of results to include relevant groups, for example, the gender perspective, minority groups.

○
There are no references to barriers and facilitators to the use of the knowledge produced by research.Overcoming such barriers or building on such facilitators are two main strategies to foster the impact of research.Yet these could be very context-specific (previous knowledge, skills, social or professional roles, values and beliefs…), and relate to individual, organisational or systemic levels.
○ 8.In their section on "Key messages", I find a major point is missing: Creation/stimulus of places for academics and practitioners to interact, work together, and define agendas and facilitators for research impact.Some academics refer to that as push-pull spaces/arenas.
In brief, I favour the indexing of this paper with some revisions.To my understanding the authors may want to include some additional published work in their paper, and reflect on other relevant points such as impact from design, impact narratives, barriers and facilitators, and academic & stakeholders'' push/pull spaces.
research approach in order to foster impact.

Is the work clearly and cogently presented? Yes
Is the argument persuasive and supported by appropriate evidence?Yes Does the essay contribute to the cultural, historical, social understanding of the field?Yes and practitioners in this space face is grappling with this challenge in describing the similarities and differences across different forms of work, which inhibits the capacity for like-minded professionals to be able to overcome their respective silos.A critical step that would enable the system to change in the way the authors endorse is to grapple with and address these language barriers.One way of addressing this in the article would be to pull in some of the other germane literatures (e.g., around the role of boundary spanners in higher education, public engagement & public scholarship, and institutional change efforts, etc.) and/or to highlight spaces in which research impacts work are already deeply institutionalized (e.g., the Cooperative Extension system in the United States).
The authors also state that "scholarship in this domain needs to be more relevant to practice"while I fully agree with this sentiment, I disagree with the premise that the primary area of scholarship that they propose connecting with practice (i.e., evaluation) will alone be sufficient to create these links.The commentary would be strengthened by acknowledging some of the other, adjacent challenges (and their associated literatures) with connecting research with practice alongside evaluation, including: Like any groups with differing expertise, researchers and practitioners in research impacts work can -and frequently do -have divergent perspectives on what questions and efforts are valuable to pursue to create a solid evidence basis.As the authors acknowledge, work that will meaningfully move the needle on research-practice linkages will necessarily require that practitioners be involved in setting research agendas, and researchers to be more intimately involved with translating their work to practice.However, funding and other supports for these collaborations are frequently lacking -better training and/or professionalization efforts to support research impacts won't address the types of resourcing gaps that keep these kinds of partnerships supported on a case-by-case basis.

2.
Different forms of expertise are needed at different stages in the research and translation process, and in general, the funding and reward systems in place around research preference discovery and/or novelty over synthesis and translation.The authors might benefit from clarifying where in the system they believe that the training and professionalization efforts they propose are likely to have the greatest impact -Is it by creating more research faculty with translation skills?In departments and university units expanding investment in professors of "practice" and other titles related to translation/impact?In deepening a commitment to professional staff to work in partnership with faculty? 3.
As the authors note, the relationship between systems of higher education and funding are critical levers as related to research impacts.The commentary would benefit from greater clarity about proposed mechanisms to evolve the funding ecosystem, especially in the context of limitations in existing funding mechanisms and allowable expenditures to support engagement and research impacts work (e.g., Can research funds be applied to paying for the effort of professional support staff?Better support the development & scaling of participatory processes?),weighed against real limits in revenue sources & priority in resource allocation.
between research impact scholarship and practice".This is a sensible recommendation and one that won't find any dispute from me as I am often advocating for exactly that.As with everything the devil is in the details, and I offer thoughts on some of those devilish details below.
The commentary lacks real world examples of where this is working well to illustrate the principles presented.For example, there is strong impact research and impact practice in the Michael Smith Foundation for Health Research in British Columbia, Canada.For many years MSFHR was led by Bev Holmes, a renowned scholar of impact.My own collaboration as an impact practitioner in Research Impact Canada/York University with impact scholars in the RIPPLE program (Faculty of Education, Queen's University) are two Canadian examples.There are certainly more.
Regarding coherent frameworks.I argue that there will likely never be a coherent framework widely accepted and practiced.In contrast I do think there are principles that will translate across many contexts, so I encourage the authors to think of principles instead of (as well as) frameworks.
Much impact scholarship comes out of discipline specific research.I encourage the authors to differentiate impact scholarship arising from specific disciplines and how practitioners can access and adapt that scholarship to different contexts.And if an impact practitioner is sitting with an institutional mandate, serving many/all disciplines, then how can s/he learn from homelessness, nursing, agriculture, etc. impact scholarship.That being said impact is emerging as it's own discipline so a reflection on research impact within a discipline and as its own discipline would help point the way forward.
But then impact practitioners run into the same barriers as other evidence end users: lack of time, lack of expertise, lack of incentive to engage with the impact evidence.I recommend adding to the last bullet under concrete measures, a nod to courses available to impact practitioners to help overcome these barriers.There are a number of courses in Canada (ie Mobilize YU from York University as one of a number) and certain there must be more in the UK and elsewhere that could add concrete examples of efforts to address these barriers.Courses will help bring the impact evidence to impact practitioners.
In the concrete measures bullet on stakeholder involvement, I think there is a missing element of impact practitioners as the stakeholders of impact scholarship.Impact practitioners are the end users of impact scholarship so a reflection on the unique opportunities for impact scholar/practitioner collaborations (see above) would ground this comment more centrally in the rest of the article.
I will explain my choice of "partly" for: Is the topic of the essay discussed accurately in the context of the current literature?
The authors cite their own work in 20 of the 48 references.This is a very high self citation rate, in my opinion.I encourage a broader survey of the literature.

○
And at the risk of being self serving (and the joy of open review since no one is blinded to the reviewers), Julie Bayley and I did anticipate this in our 2019 paper in Emerald Open Research in the description of an impact practitioner who can be considered advanced in their research impact literacy.We described this as being "critical", defined as "Critically engaged with the evidence, understands there is a body of expertise, knowledge and tools which ○ can underpin practice and is able to (i) synthesize, (ii) critique and (iii) add to/extend it.Likely to be able to comprehend at a strategic and/or systems level".An impact practitioner who is critical is already at the nexus of impact scholarship and impact practice.Reviewer Expertise: research impact, knowledge mobilization, institutional research strategy I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

○
Is the topic of the essay discussed accurately in the context of the current literature?PartlyIs the work clearly and cogently presented?YesIs the argument persuasive and supported by appropriate evidence?PartlyDoes the essay contribute to the cultural, historical, social understanding of the field?YesCompeting Interests: No competing interests were disclosed.

©
2023 Phipps D. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.David Phipps Division of Vice-President Research & Innovation, York University, Toronto, Canada I have no further comments on version 2.

References 1 .
[https://emeraldopenresearch.com/articles/1-14] Bayley J, Phipps D: Extending the concept of research impact literacy: levels of literacy, institutional role and ethical considerations.Emerald Open Research.2019; 1. Publisher Full Text Is the topic of the essay discussed accurately in the context of the current literature?Partly Is the work clearly and cogently presented?Yes Is the argument persuasive and supported by appropriate evidence?Yes Does the essay contribute to the cultural, historical, social understanding of the field?Yes Competing Interests: No competing interests were disclosed.

the topic of the essay discussed accurately in the context of the current literature? Partly Is the work clearly and cogently presented? Yes Is the argument persuasive and supported by appropriate evidence? Partly Does the essay contribute to the cultural, historical, social understanding of the field? Yes Competing Interests:
The development of a Co-creation Impact Compass for healthcare researchers.PLoS One.2020; 15 (10): e0240543 PubMed Abstract | Publisher Full Text 4. Stier J, Smit SE: Co-creation as an innovative setting to improve the uptake of scientific No competing interests were disclosed.

have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
https://doi.org/10.21956/openreseurope.16766.r34252 That there are real, important differences across disciplines with what 'scholarly rigor' entails and how this relates to the types of impact goals and efforts most commonly associated with work in those disciplines (e.g., Doberneck & Schweitzer, 2017; Fecher & Hebing, 2021).This challenge is exacerbated when different types of scholarship are valued differently, both within and across fields, which can lead to experiences of epistemic exclusion which differentially impact scholars with marginalized and underrepresented social identities (e.g.,Settles et al.,