Measuring Research Impact in a Health Service Is a Worthy but Complex Goal; A Response to Recent Commentaries

<jats:p>
          </jats:p>


W
e thank the authors responding to our recent article describing our research impact evaluation at a regional Australian hospital and health service. 1 The five considered commentaries contribute to the continued and much needed dialogue about embedded research integrated into health systems.We deliberately use "embedded" and "integrated" to infer research is fixed firmly (embedded) and incorporated (integrated) into health systems so that research in health systems achieves scholarly rigour and importantly practice change.In response, we address each commentary individually and provide context for our health service moving forward.
Ramanathan argues the crux of an impact evaluation is to report the benefits and impacts of the research undertaken, and rightly note this evidence was difficult to identify in our study. 2The impact examples in our study were identified through interviews and document review, leading us to recommend Townsville Hospital and Health Service (THHS) and similar health services, prospectively and systematically collect impact data including patient outcomes alongside traditional research outputs.Systematic collection will facilitate increasingly robust research impact evaluations.Ramanathan states our study measured research activity and productivity, not impact; and suggests field weighted citation indexes could have been reported.We agree, while also emphasising that healthcare delivery organisations must also identify and collect relevant clinical and workforce-related indicators of research impact.
We thank Abramo and D' Angelo's for bibliometric analysis of THHS data and are pleased to note the increasing trends from THHS. 3 The scholarly impact measured by bibliometric analysis is certainly an important aspect, but again we emphasise the importance of looking beyond measures of research activity to indicators of impact that incentivise knowledge translation pathways into clinical practice.Future efforts to define impact for our healthcare organisation will align with the social impact they describe.
Williams and Fernandes surmised three lessons from our study: (1) achieving a shared definition and expectation of research, (2) the importance of stakeholder engagement, and (3) enabling research across a system. 4They correctly noted the diverse expectations of research among key informants at THHS which can (and should) be ameliorated by stakeholder engagement with management to clarify the purpose of investing in research.We wholeheartedly agree on the importance of stakeholder engagement, particularly managers and community, to support the translation of evidence into practice.As Williams and Fernandes state, without a shared definition and expectation of research, research impacts will be equally hard to define and measure. 4We also agree with the suggested approach for individuals from all clinical and management disciplines to develop research skills such as critical appraisal, community engagement and analysis of financial cost and health utilisation; noting we have made recent progress at our health service.
Hanney commented on the specific and broader impacts identified in their evidence synthesis on research engagement and healthcare performance.The specific impacts refer to rapid implementation of research which we consider the most relevant for research conducted in healthcare organisations. 5Hanney argues the research successes and challenges during the COVID-19 pandemic demonstrate the importance of embedded research within health systems.At THHS, COVID-19 had an overall negative impact on research, with an institutional pause on research (except COVID-19-related studies or clinical trials of limited treatment options).Researchers and research administration staff were re-deployed to the COVID-19 front line creating a backlog, particularly in ethical and governance approvals.This shift in resources underscores the importance of integrating workforce and infrastructure investments into efforts to embed research in a health service.We also call for improved integration of research and healthcare, with appropriate resourcing, to provide better patient outcomes; but highlight the regulative barriers often faced by health service administrators who are accountable to healthcare