A Framework for Assessing and Improving Decision-Making in the Translation of Research and Innovation for Impact

The successful translation of research and innovation into impact has never been more important to solve the complex problems facing societies and existential threats to our planet. University research and innovation holds a key to unlock solutions. Innovation practitioners and university research translation offices seeking to support and foster promising innovations face a challenge in determining which innovations have sound prospects for development through to the market. Approaches such as mission-oriented innovation; as well as open innovation, entrepreneurial universities, and educating the educators considered in previous letters all may help, but how can we be sure that our assessments of any given initiative are sound? What are the requisites for successful translation to impact, and what tools might aid in decision making for complex endeavours? There are many theories, tools and methods in use; yet the performance of research translation efforts at universities around the world vary wildly with only a very small number clearly successful and the vast majority, achieving at best, modest results, that are well below their expected potential. We assembled a multidisciplinary team bringing together research translation expertise and innovation researchers with a record of entrepreneurial success to consider tools that can aid identification and assessment of promising innovations, with a view to best aligning university support. Reflecting on existing translation readiness assessment tools and practitioner knowledge, we argue that a more encompassing and dynamic framework is needed to aid universities in early and ongoing assessment of innovations to effectively steward their development and enhance their potential for success. We provide an early outline of this multi-dimensional assessment tool.


Introduction
With humankind facing global challenges of unprecedented complexity and growing urgency, such as the sustainable development goals and targets expressed in the Agenda for Sustainable Development by the UN (2017), the success of research translation and innovation could never be more important.New approaches such as mission-oriented innovation and design thinking in the OECD (Larrue, 2021) and many topics raised in previous letters -including open innovation, the entrepreneurial university, the civic university and educating the educators -have been implemented or proposed.Yet, according to WIPO (2022) the social impact of innovation continues to be relatively low, reflecting under-performance in innovation, translation, adoption, and diffusion with indicators pointing to systemic problems meeting expected targets.The Global Innovation Index (GII) report (WIPO, 2022), while optimistic about academic publications growth, is pessimistic about innovation, translation, adoption, and diffusion.The report reflects that the social impact of innovation is at a low point and that all proxy metrics indicate the impact of innovation is slowing down, with productivity growth at its lowest level ever.
Due to the diversity of each nation's research and innovation culture and ecosystem, this can mean different things and necessitate different responses, and of course, basic research takes more time to translate than applied research.In Australia for example, according to the Productivity Commission (2023) report, "Some 98% of Australian businesses do not produce new-to-the-world innovations.They are adopters, adapters, incremental improvers.Supporting them to take up new technology or adopt a business innovation could have profound and broad productivity benefits."For Australia then, translation effort into diffusion of innovation is a priority.For other countries, the priorities may be different.However, even for the translation of new-to-the-world inventions and innovations, globally, the failure rate of start-ups and initiatives remains high with, "Innovation research in the past decades [failing] to deliver clear and consistent findings, coherent advice to managers, and convincing 'best practice' solutions" according to Van Oorschot et al. (2018, p.1).Further, and despite impressions fostered in the media built on high profile stories of a very small number of companies that achieve unicorn status, "more venture-backed start-ups fail than succeed, and venture capitalists (VC) themselves aren't much better at generating returns.For more than a decade the stock markets have outperformed most of them, and since 1999 VC funds on average have barely broken even" (Mulcahy, 2013).Indeed, it appears that VC investment decisions are often predictably bad (Davenport, 2022), and although research organisations can celebrate when their spin-outs have raised millions of dollars, this is not a good indicator of the probability of success.Yet, it is worrying that the success rate of research and innovation translation, including commercialisation by universities, is so low.This is a clear indication of an unmet need for understanding the drivers of success, the causes of failure, and the necessary tools to improve.
While acknowledging the primary mission of universities is education, they are also in a prime position to influence the successful translation of new knowledge for positive societal impact.At our university, the concept of "translation readiness" to guide the achievement of impact has been used since 2016 based on a rudimentary framework by co-author Mak to assess seed-funding applications.

Insights from a Review of Translation Decision Aids
In stewarding development of innovations, universities and their partners or collaborators need to consider both viability and impact.They need to discern which projects have most potential for success, and requirements throughout the translation pathway to realise this success.Tools and methods to aid in assessing the likely success of new innovations, research translation projects, and start-up ventures are needed.
We conducted a literature review which uncovered a plethora of theories, tools, and methods both in use and in development, from Technology Readiness Levels (TRLs) first introduced by NASA in 1974 noted by Héder (2017), to the Unified Theory of Adoption and Use of Technology http://www.open-jim.orghttp://creativecommons.org/licenses/by/3.0II (UTAUT) introduced by Venkatesh et al. (2003), to Evolutionary Choice Theory (ECT) proposed by Stroh et al. (2023).The demand has been driven by a variety of factors including: increasing pressures on universities and research organizations to generate demonstrable returns on investment for society; the growing number of venture capitalists; and the growing certainty that sustainable survival for all enterprises is dependent on successful innovation.These factors are universally amplified by the continuing poor results generally achieved using these available tools, despite their often wide-spread acceptance and in some cases decades of associated research.
Numerous tools have been developed to assess progressive stages of innovation readiness through to market adoption.The traditional TRL model focuses on development of the innovation "product", though is limited in that this is isolated from external environmental factors such as market demand and value, user readiness and acceptability, as well as internal factors, such as the competencies, motivations and innovation experience of the research team.Other layers of readiness level have been identified to address external contingent factors, notably commercialisation readiness (CRL), market readiness (MRL), regulatory and legal readiness (RRL and LRLs), operational readiness (ORL) and societal readiness (SRL).Recognising that technology development, commercial viability and uptake are influenced by a confluence of these factors, in recent years we have seen the emergence of integrated assessment frameworks such as the Technology Translation Readiness Assessment Tool by Zorilla et al. ( 2022), the Commercial Readiness Index developed by government (ARENA, 2014) and Balanced Readiness Levels by Vik et al. (2021).Within information sciences, a prolific body of research exists based on the Technology Acceptance Model (TAM), its precursor Diffusion of Innovation, and its progeny including the Universal Theory of Acceptance and Use of Technology (UTAUT) and Active Innovation Resistance (Davis et al.,1989); (Joachim et al.,2018); (Rogers, 1995); (Venkatesh and Bala, 2008); (Venkatesh et al., 2016).Innovation researchers have also identified a broad variety of relevant variables and moderators impacting each activity undertaken within innovation efforts.We have not surfaced evidence to establish that any of the frameworks or methods identified are generally effective or specifically useful for practitioners seeking to pick winners or optimize the allocations of resources to support research translation, innovation efforts within enterprise, or start-up success.
While our work is ongoing, a number of conclusions have been established.Most prominent amongst these is the absence of empirical evidence supporting the generalized use of available tools.Especially needed is further longitudinal research empirically establishing the usefulness or effectiveness of any of the existing assessment methods for generating useful insights into the actual sustained viability of research translation, innovation, and start up opportunities.For example, Hasenauer et al. (2016) represents the only longitudinal study related to "translation readiness", we have found to date.Surprisingly, while some examples of sound and practical tools were discovered for specific forms of assessment, despite the widespread use of many methods, there is very limited evidence supporting the veracity of any of the most widely used.There is also substantial evidence establishing that available methods are incomplete for pragmatic generalized use by universities of all kinds, enterprises, governments or venture capital firms.
That said, our review has surfaced reasons to consider some tools and methods more closely.What is clear, however, is that substantial research establishes a more holistic approach and resultant framework is required.This includes research establishing the significant influence of psychological factors and cognitive decision processes on both the selection or resistance to proposed innovation and new venture programs as well as adoption or resistance to their outcomes (Bhimani et al., 2022); (Roberts et al., 2021); (Stroh et al.,2022;2023).
In addition to the clear need for more research and specifically longitudinal studies, we have also drawn a variety of conclusions that appear to be of high value.In particular, there are strong http://www.open-jim.orghttp://creativecommons.org/licenses/by/3.0III indicators that several long-standing core foundations need to be revisited.Existing approaches generally share three common assumptions.First, they focus on assessment at specific points in time.Second, they assume a linear process where opportunities progress through stages in a sequence or levels.Finally, they focus on observable variables or characteristics of ideas, technologies, individuals, teams, organizations, markets, or policies.
One of the clear contributions of involving practitioners and executives in the translation process is clarity that the characteristics are explicitly only relevant in context and the determinant of success or failure are not any particular set of characteristics as proposed by tools such as TRL, but rather the interaction between such elements.In effect, product readiness can only be determined in context of an increasingly dynamic customer need.As highlighted by methodologies such as Lean, the effectiveness of a solution in satisfying a need does not in itself result in success.The way in which the solution is presented to the user and buyer, the effectiveness of the value proposition and whether it triggers resistance, can all equally be determinants of success.This importance of assessing the interaction between elements is supported by Active Innovation Resistance research according to Joachim et al. (2018), research on the role of psychological factors (Stroh et al., 2022;2023), and research comparing Open and Closed Innovation processes based on the Technology-Market Readiness tool (Hasenauer et al., 2016).
The characteristics of a team may be viewed in a similar way.The skills and capabilities to lead and manage a start-up team of five members are different from those required to lead and manage an organization of fifty or one hundred and fifty.Thus, the interaction between elements, psychological factors, the mix of skills and experiences required at different stages for different types of activities is the determinant of success or failure more so than any assessable "level" or observable characteristics of one element at a point in time.
The very concept of "readiness" itself may in fact be a barrier or a conceptually problematic starting point.Given the composition of skills required by a team at different stages is different, readiness cannot be assessed for a technology or research outcome only for a group of people.Further, as these necessary capabilities and perspectives change, they are not necessarily assessable except and until the need for particular activities approaches.This may explain both the over importance placed by VCs on the founder characteristics according to Davenport (2022) and its ineffectiveness as the problem or need and technology or solution fit must be assessed separately to the value proposition, market or customer fit.
The barriers and reasons for failure need to be assessed differently from the prerequisites of success or the characteristics that are associated with success.Many failures have all of the identified characteristics shared by the successes and yet failure still occurs.Many causes of failure are avoidable but are contextually specific and must be specifically mitigated or addressed irrespective of the presence of characteristics common to success.This is explicitly supported by research on technology adoption or resistance to new technologies or products by Bhimani et al. (2022); Joachim et al. (2018); Kuhn (1962); Sjödin et al. (2019); Stroh et al. (2022Stroh et al. ( , 2023)); Watve (2017) that makes clear that not triggering resistance is as important to success as offering a rational case for adoption.
In summary, we recognise that innovation success is more likely when: a) The innovation addresses an acknowledged need with a clear sense of the end user and value, together with a plan to negotiate market, regulatory and other considerations of uptake.b) The researcher, project leader, or team has an entrepreneurial mindset and associated skills, or a track record in developing innovations through to market.
http://www.open-jim.orghttp://creativecommons.org/licenses/by/3.0IV c) The organization or research institution has the commitment, resourcing and expertise to support advancement along the translational pathway and facilitate optimal routes to uptake and impact.
Requirements for a new proposed framework and decision aids are clear.These must be parsimonious, inform practitioners as well as researchers, offer consilience across different relevant disciplines of research, and satisfy the explicit need for indicators required by innovation managers and organizational executives.A new class of measures may be required.First, to overcome Goodhart's Law, that is, the tendency for measures ceasing to be good measures once they are targets.Second, to accurately capture and reflect the necessity to assess the "fit" or interaction between dynamic elements rather than characteristics of elements.And third, to ensure focus on the perceived requirement to assess and build people, skills, and decision making as much as traditional parameters of research outcomes, technologies, or products.

Framing the Innovation Odyssey
Rather than conceptualising progress in terms of levels or sequential stages, we recognise the non-linear and often cyclical or iterative nature of research and innovation translation, which begs the need for a new metaphor.The term "valley of death" has found widespread adoption to characterize the resource gap between research and impact, especially for technologies.The term conjures barren images of Nevada's Death Valley, yet the translation may be neither in isolation nor obligated to go through barren wastelands.We propose instead that translation be better likened to Homer's Odyssey.Re-situated as a human journey, it engenders hope that despite many trials over many years, like for Odysseus, through leadership perseverance, ingenuity, navigating obstacles, and drawing support from partners and allies; researchers and industry can ultimately achieve goals, make meaningful impact on society and find reward.
We propose a holistic framework to assess translation readiness informed by four intertwined strands: the Innovation, Beneficiaries and Socialisation (i.e.translational environment), the Team, and Organisational Support.Drawing on the Homeric analogy, the Innovation is the Homeric galley passing through challenging seas (translational environment) to reach its destination (Beneficiaries); Beneficiaries and Socialisation is represented by the encounters along the way, to finding "home" that represents acceptance and uptake.The Team encapsules the skills, motivation and courage of the leadership and "crew", while Organisational Support provides stewardship throughout the journey: the favourable wind to maintain momentum (translational resourcing and capability development) and guidance towards the desired destination (impact and commercialisation pathways, commercial partner attraction/start-up formation).In our framework, a product is any form of research or innovation output, the market is any form of beneficiary prepared to adopt, use, or pay for that product (noting each of these may be different), and the fit relates to socialisation or the interaction of the product's set of characteristics, its presentation to prospective beneficiaries, and the interaction of those with the beneficiaries' decision processes, and thus their adoption or the triggering of resistance.
Notably, this framing allows universities, partners and collaborators to allocate their resources efficiently to drive translation and success.In this framework, certain translational projects may be more developed in some categories than in others, which acknowledges that different teams, innovations, and target users can require different support at different times.This is in contrast to the linear scales that many existing approaches have, that do not inform triaging, improvement http://www.open-jim.orghttp://creativecommons.org/licenses/by/3.0 the level of competition, etc.Although this process is well-known and touches on key topics for all entrepreneurial programs, this holistic approach is poorly practiced, especially in research organisations.Our approach is flexible and modular, acknowledging that these various factors are important, but most importantly, they are interconnected, and therefore we don't define all factors and connections here.We also acknowledge that different forms of research require different activities in order to successfully translate their results to impact.As such, different components of the framework will have varying degrees of importance or may not be relevant to specific contexts or translation pathways.

Beneficiaries & Socialisation
As discussed, a few examples of sound and practical tools were discovered through review that fit modularly into our framework.These include the Technology-Market Readiness (TMR) tool proposed by Hasenauer et al. (2016).The TMR relates to the "tech" and "market" sub-dimensions of our framework and reports on a longitudinal study of 57 start-up firms over 18 months, with a focus on those engaged in social or environmental entrepreneurship with multiple technology types, motivated for combined economic, social and environmental impact.Comparing open and closed innovation approaches, they found that in the open incubation process, fellow start-up firms and external experts identified errors early and accelerated their incubation, achieving higher technology readiness levels and lower technology risk, especially in the life sciences.Zorilla et al. ( 2022) developed a technology translation readiness assessment tool (TTRAT) for entrepreneurs and innovators that touches on multiple framework dimensions.This was developed for the disability industry, where they mean "technology" to cover software, devices, instruments, standards, processes and clinical guidelines, needed by an estimated billion individuals worldwide to live active, independent lives.The instrument poses questions around: the problem, stakeholders, solution, benefits, competition, differentiation, team, sustainability, and technology.Finally, Stroh et al. (2022Stroh et al. ( , 2023) ) provide details of psychological factors that must be assessed both within organizations considering opportunities and which influence adoption and resistance of outputs by users.As we complete our SLR and analysis, we look forward to providing a more comprehensive description of a holistic framework and each of its components.

Concluding Remarks and Future Directions
To support significant innovation initiatives (e.g.mission-oriented innovation, the entrepreneurial university, the civic university), strategies like open innovation and educating to innovate, and funding schemes such as Horizon EU, we need to know that the right settings are in place for successful innovation in complex settings.We have outlined a framework to aid innovation http://www.open-jim.orghttp://creativecommons.org/licenses/by/3.0VII interventions that covers key dimensions of readiness throughout the innovation journey, and described a process to help with the right interventions at the right time that captures the data to inform longitudinal studies.Universities are well positioned to assess the effectiveness of translation approaches and assessment models over time, and this will provide a much-needed evidence base and rigour to validate translation assessment tools and inform university approaches to fostering successful innovations.We illustrated the application with a few sound tools from the literature.But more is needed.Sound tools are few, and open decadal-length longitudinal studies from beginning to end of failed and successful innovation translations that include not only economic data, but the kind of contextual factors illustrated in our framework, is a prerequisite to understanding what works in innovation and what doesn't, and we believe this framework is a meaningful step towards allowing universities to collect this data and act on the findings.

Figure 1 .
Figure 1.A framework for innovation translation readiness