Reflections on the Researcher-in-Residence model co-producing knowledge for action in an Integrated Care Organisation: a mixed methods case study using an impact survey and field notes

: Background

services come under increasing pressure to re-design services now rather than wait for research.One such approach is the 'Researcher in Residence' (RiR) model which seeks to co-produce knowledge for action.The aim of this paper is to extend the evidence base regarding mechanisms of impact.

Methods
A two-year mixed-method case study of the experience and impact of two part-time RiRs, embedded within an Integrated Care Organisation to support the implementation of new models of care.Data included the results of an anonymous impact survey sent to 80 key stakeholders, field notes of meetings (n=112), and observations of naturally occurring events (n=68).

Findings
Impacts were identified in relation to use of co-produced evidence, capacity building, changes in ways of working, and to a lesser degree changes in operations or strategy.Impact involved learning which was mediated by three non-linear, non-predictable, positive and negative feedback cycles (learning/improvement, access, expectation).A mixture of technical skills, personal attributes and behaviours were identified as key to this mediation.

Discussion/Conclusion
The RiR model promises a timely, applied and transferable research model that contributes to the development, evaluation and adaptation of innovations that seek to integrate services where the evidence base is weak and uncertain.However, the model is not without challenges.These could be addressed by flexibility of research design and funding, and adequately supporting and developing key attributes of RiRs.
Appendix 1 -RIR Impact Survey Over the last 24 months the [Funders name] Fund, in conjunction with the [name of organisation], has funded 2 part time Researchers-in-Residence to help research & evaluate the care model.
This approach to research is novel as it embeds the researcher to co-locate with teams in the organisation, even though they are employed by the University.This should mean that the researchers are better placed to give advice and guidance to us as we develop the model as well as improve use of evidence, working in close proximity and developing relationships with key stakeholders.
We want to undertake a 360° review of the role and would be grateful if you can answer the following questions.Your responses will be treated confidentially and data will be analysed anonymously.This should take approximately 5-10 minutes."excellent" (1) or "good" (4) communication skills; engage with "wide" (2) or "broad" (1) range of "people"/"professionals"/"Staff"/"public'; Communicate in "in laymen's terms"; "Academic and health 'language' is very different and can result in misunderstanding and disengagement"; Presenting in "engaging format"/ "reports are both academic and corporate"; Relationship Skills (13 codings): Ability to build relationships (4); Change Management (4); Being Approachable (3); Ability to connect (2), Ability to influence and persuade (1) "Build a good rapport with staff"; "Be leaders of change"; "bottom up approach"

Specific experience to be acquired while embedded (10):
Experience and knowledge to be acquired locallyembeddedness, relevance, understanding "Understanding of local context" (2); "They need to spend some time directly working with some of the team to better understand the wider issues that we face"; "a broad understanding of the environment that they are called to research in and the problems that it is facing , so better equipped to engage and deliver relevant research"; "good understanding of our local system priorities and their role in supporting the delivery of these"; "more understanding of the context -difficult to put into words, but ability to see the larger picture and not just the aspect they are working on?";"Knowing one another and the organisation and the systems, brings greater depth and richness to projects and ultimately this will improve outcomes and impacts with more value" Enthusiasm (4); Confidence (3); Patience (1), Perseverance (1) "Enthusiasm by the bucket full"; "a belief in what they are doing"; "confidence to voice sometimes unpopular messages when feeding back"

Emotional Effort
Rare/occasional exposure to distressing or emotional circumstances Motivation and Passion (3); Energy (2); Empathy (1); Humour (1) "Motivation and passion"; "good energy"; "I really welcome his refreshing and uplifting approach to his work"; "empathy with their audience"; "A sense of humour"

Introduction
The translation of research-based knowledge into practice -knowledge mobilisation -has become the focus of policy makers worldwide, as health and care services struggle to cope with rising demand, greater patient expectations and stagnating budgets.How to bridge the "know-do" gap has led to the emergence of new academic disciplines, such as Implementation Science.Typically, this portrays the implementation gap as a failure of the delivery mode, front-line staff or institutions to use the latest knowledge, a problem that can be fixed by studying the factors that facilitate or hinder models of evidence roll-out (Greenhalgh 2017).To date and internationally, the impact of efforts to understand and improve such knowledge transfers appear to have fallen short of expectations (Kreindler 2016, Lockett et al 2014, Reed et al 2017).
In integrated health and social care services, the translation problem is exacerbated by the fact that the evidence base to support many proposed changes, may be weak methodologically, ambiguous in its findings or absent all together.A case in point is the evidence-base to support the UK policy (National Audit Office 2017).Many studies lack conceptual clarity, are poorly designed, or operate in very different health service contexts limiting their generalisability (Damery 2016).
Such concerns have given rise to a shift in models of knowledge mobilisation from passive, linear and instrumental conceptions that see 'knowledge' as a transferable product; through a more relational approach connecting researchers and decision-makers; to the idea that knowledge is neither fixed nor privileged (Davies et al 2015, Ward et al 2009).Reflecting the growing emphasis on the 'Zeitgeist' of coproduction (Palmer et al 2018), embedded researchers such as "Researchers in Residence (RiRs)" have gained traction (Marshall 2014, Marshall et al 2016, Marshall et al 2014), supported by international calls to action (Holmes et al 2017).Researchers in residence can sit in the 'middle ground' (Guthrie et al 2017) between research and evaluation.In our case, the model combines embedded, operational service evaluation with an action-orientated, participatory approach to research that seeks to impact the development and implementation of service innovations in real time.
The Researcher in Residence (RiR) model seeks to co-produce (Clarke et al 2017) 'knowledge or theory in practice' (Argyris andSchön 1974, Van de Ven 2007).To achieve this (Vindrola Padros et al 2017) suggest Final manuscript (NOT anonymised) researchers should: 1) be co-located with or embedded in the services for significant amounts of time and have a stake in the research or improvement initiative they are supporting, and (2) negotiate the meaning and use of research-based knowledge and co-produced knowledge that is sensitive to the local context and responsive to the continuously changing needs of the system.
However, the evidence base to support this model is limited, comprising a few case studies (Eyre et al 2017, Lalani et al 2018) and a narrative review (Vindrola Padros et al 2017).These have sought to clarify the model and identify challenges such as relationship building, flexibility around the scope of the work, and tensions with maintaining professional identity (Vindrola Padros et al 2017).These have mainly been based on experiences of RiR teams rather than the partners they work with.Furthermore, questions still remain about mechanisms of how the RiR model impacts on stakeholders and systems.This paper reports on the impact of two RiRs working on care model innovations in an integrated care provider organisation, as perceived by stakeholders.It seeks to identify attributes and behaviours of the RiRs and stakeholders that facilitate effective interaction and several key processes important to delivering impact, laying foundations for theory development.

Setting
In Torbay and South Devon, the acute NHS Trust and adult social care and community services merged in October 2015 to create an Integrated Care Organisation (ICO) -among the first in the UK to adopt this new organisational form -implementing a new care model, as part of the integration of services.This vertical integration builds on the history of horizontal (i.e.across sectors) integration between organisations working in the Torbay area (Wilding 2010) The RiRs were mixed methods specialists and mid-career research fellows, including a public health consultant with analytics skills and a social scientist with ethnographic skills working collaboratively on all aspects of the projects.They were supported by the QI Lead and holding regular meetings off-site with the principal investigators, a health policy researcher and a GP/primary care researcher (all co-authors).
Governance and meaningful co-design was also assured through Executive sponsorship by the Deputy Medical Director and regular attendance/reporting to the bi-monthly cross-system 'Care Model Delivery Group'.
The RiRs have been working together with managers and frontline teams to support the evaluation of specific care model service innovations, the development of a metrics dashboard as well as conducting research into how these and other service innovations integrate to impact on person-centred, coordinate care (PCCC) (Lloyd et al 2017).The research was a mixed methods study involving a phased, iterative process of collecting and analysing quantitative and qualitative data with the aim of developing mid-range theories (to be reported elsewhere).The core of the RiR activity focused on two care model innovations 1. the enhanced intermediate care service (EIC) provided by multidisciplinary teams in the community looking to avoid hospital admission or bridging journeys back to people's homes, and 2., and the co-located holistic link-worker/social prescribing, Wellbeing Coordinators Programme(WBC), ideally preventing people to become unwell in the first place.The scope also included a case study of how these contribute to integration in one of the five ICO geographical locality hubs.

Methods
This paper is based on data from a stakeholder on-line impact survey and reflective field notes collected by the two RiRs during the first two years (April 2016-2018).
The survey design was based on impact domains identified through three publications by the department of Quality Improvement (QI) at the University College London (Eyre et al 2017, Holmes et al 2017).It was developed by the two RiRs and the QI lead (co-author) with input from Principal Investigators and tested with two further colleagues.The purpose of the survey was two-fold.First, it was to explore the impact of the RiRs and through this to develop a business case for continued investment in researching and evaluating the care model as it develops.Second to more fully understand the mechanisms of impact and attributes required of a successful RiR model.The survey contained nine questions, of which six contained 5-point Likert Scale response options.The six quantitative response options captured: i) overall impact; (ii) new ways of thinking; iii) evidence use; iv) up-skilling staff; (v) impact on operations/planning, and vi) impact on organisational strategy/culture.As well as open comments boxes for each quantitative question, there were also three qualitative questions asking: where the RiR approach worked best, what might be improved, and the required attributes of a RiR.The survey was conducted anonymously, but respondents were invited to identify their job role in the last question (see Questionnaire in Appendix 1) The on-line survey was sent to 80 stakeholders who were recorded as active partners through the matched funding capture system and deemed to have engaged meaningfully with the RiRs.The sample comprised a mix of professional backgrounds and roles from commissioning, primary, community and acute services, social care, voluntary services, and patient and public representatives.Noting that some had mixed roles as health professionals and managers, the distribution of main job roles in this sample was: 44 managers, 16 health professionals, 12 voluntary sector representatives, 5 administrative staff and 3 patient and public representatives.The heterogeneity and overlap of stakeholder attribution with various evaluation projects (both service innovations were co-located in community hubs) made it difficult to look for meaningful quantitative response patterns by potential sub-groups.The survey ran from April 2018 and was open for three months.Two automated reminders were sent, and non-responders followed up via email or verbally by the QI Lead.
Reflective field notes were recorded independently by both RiRs.Typically, notes were made following meetings and written up in Word, and stored on a secure, password protected server using NHS devices and servers.Notes were related to dedicated meetings (n=112), or at naturally occurring events (n=68).
The analytic strategy collated the quantitative results of the survey and interpreted the findings in light of: a) an analysis of the qualitative survey comments (using basic content and thematic analysis in NVIVO and Excel, analysing prevalence of codings for three qualitative questions in particular); b) automated search queries run on the field notes in NVIVO; c) comparing and contrasting the factors that RiRs felt influenced their impact in three selected project areas; and d) triangulation of data sources, iterative sense-checking of categories and themes, discussion and reflection between co-authors. .

Findings
The following section comprises quantitative findings of the stakeholder online survey, which helped build an understanding of how RiRs brought about change through their activities.The section then goes on to illustrate identified impacts through a range of feedback cycles.

Impact survey:
Fifty stakeholders responded to the on-line survey, a response rate of 62.5% (N=50/80).Qualitative comments were reported from a minimum of 22 respondents to a maximum of 42 respondents for respective open comment boxes and the three qualitative questions.Of the 50 respondents only 35 stakeholders and partly disclosed their role, which made it hard to find and analyse response patterns based on possible sub-groups.
Figure 1 shows that over half of respondents (53%) reported a great or moderate impact on 'Improved use of evidence (research findings, guidelines, dashboards, evaluation data etc)', which was better than expected.Respondents reported a similar level of impact on 'Encouraging new ways of thinking/working, e.g. on their perspectives, work or practice', as well as on 'Improved research and evaluation skills (i.e. through advice, practice or training)'.Impact perceived as going beyond the RiR's influence on the individual, such as on 'Influencing strategic direction or culture (communications, business cases, organisational learning)' was less at 44% of stakeholders reporting a great or moderate impact, and lesser still when respondents considered the impact on 'Influencing operations and planning (MDTs, care delivery etc.)' (39% great or moderate).'Figure 1 Here'

Qualitative analysis:
Key to facilitating impact were three interacting, positive and negative, feedback and feed-forward cycles: Expectations cycles emerged through managed and negotiated input, and were sensitive to organisational structure and sector; Access cycles were grounded in successful and widening engagement and relationship building; and Learning and Improvement cycles facilitated a process of mutual learning about the RiR approach and integrated care, that assisted partners to co-create impact directly and indirectly at the level of individuals, teams and organisations.This included impacts generated on a continuum of attribution, which were more direct on individuals and teams (in terms of capability and capacity building) and more indirect in relation to operational planning, strategy and organisational culture.
The interaction between cycles must be considered non-linear, and impact was not necessarily dependent on each cycle, e.g.learning could happen at any stage of the process or aligning expectations might lead to learning and improvement.The dynamics of each of the cycles and their relationship with learning and improvement, and RIR attributes are discussed below.

Expectations
The RiR contract and research questions developed to satisfy both the ICO and the research funding body, were broad in spectrum, in part recognising that the care model was new and would develop over time.This provided flexibility to iterate and co-design the evaluation and develop the research questions.However it also made managing expectations challenging.expectations evolved over time and sometimes differed in relation to the scope and focus of the work, the approach and role of RiRs, clarifying research and evaluation aims, and the required resources and capacity to deliver.Resolving these differences required an iterative process of dialogue, negotiation and compromise.Alignment of expectations facilitated engagement to increase potential for new ways of (joint) working and mutual learning.This management and negotiation worked better through increasing trust between the RiRs and stakeholders, and where expectations were un-resolved or conflicting, disengagement could arise.Repair was possible through more engagement in further activities to build trust.It could also result in work opportunities and activity, increased through its intersection with the Access Loop.

Scope and focus
The original research and evaluation protocols identified four areas of evaluation as well as a system-wide piece of research on implementation.It became apparent early on that the focus of the research and evaluation were too broad and ambitious, given the resources, time and capacity available to the RiR and stakeholders, and the emerging, iterative approach the RiRs were taking.
Differing expectations on the scope of research, which initially included system level implementation was also perceived to be uncomfortable for some senior managers.As one middle manager commented:

I think the impact issue is not about the RiRs themselves, but about what the strategic management is willing to hear from their reports (good, bad and ugly) and recommendations and what improvements the management team wants to implement.
This on-going debate became most apparent at the 12-month reporting stage and led to renegotiating the focus of the research on one geographic locality and reducing the number of evaluations of innovations to focus on the two with most novelty and investment (EIC, WBC).

Approach and role
When the funding was applied for, the aim from the beginning was to try a more novel approach to research, using the RiR model.However, the host organisation and the team appointing the posts had little experience of this role and therefore it took time and learning from all parties to embed the role.This was, perhaps, no surprise given the absence of an accepted blue-print for the model.Nevertheless, these differences had to be negotiated at several stages.
As one senior manager, driving the care model developments noted: More clarity on role at the beginning would have helped -although it was an iterative process of understanding together so it's important to have enough time to do this

Resource and capacity
Survey respondents also highlighted differences in expectations related to resources and capacity to undertake joint evaluation activities.The participatory approach placed more demands on managers and front-line staff to commit time and resources to evaluation at a time when the organisation was committed to implementing its service innovations 'at pace and scale' ('Extraordinary Joint Management Meeting, Observation note 005, 16.05.16;'All Manager Meeting', Observation Note 019, 25.07.16).
The difficult trade-offs between the desired pace of delivery, mobilising internal resources to support the evaluation, and scientific rigour was captured by this survey respondent:

Access
Access to and building relationships with multiple-stakeholders in the health and care system was crucial to undertaking evaluation and research activities, particularly as the focus of the RiR's work was on integrated care horizontally and vertically.Spending time engaging with or embedding in different parts of the system formally (i.e. through co-location) and informally (i.e.supporting related evaluations beyond the host organisation) enabled the RiRs to build networks and develop trusting relationships, principally through generating capacity and sharing expertise.This in turn enabled the RiR to identify championing leaders or staff.In several instances, this circular process may have led to an increase in engagement between the host and the wider community.
A key to access was acquiring honorary contractual status with the ICO.An outwardly visible effect of this, was the issuing of NHS email addresses as well as blue NHS lanyards, including badges with RiR names and Dr-titles.These were used always while engaging with NHS stakeholders to demonstrate insider status, and more selectively used when engaging with non-NHS stakeholders, where RiRs might use University lanyards to express credibility and independence (Observation Note 059, ICO Board Away Day, 13.07.18,RiR: "I wear this badge with pride").

Time to build relationships and expand networks
A crucial aspect of gaining access was the RiRs' ability to spend time with people in different parts of the system.Over the first two years, the RiRs engaged with 124 stakeholders (totalling 874 hours of coproduction) across the system (as active partners not research subjects).This enabled the RiRs to identify relevant stakeholders, including the public, and to build working relationships and a 'platform to influence' (Stakeholder survey: Senior clinical manager).This happened mainly through supporting senior clinical managers on service evaluation or system transformation.Effective working relationships were also strengthened when the RiRs formally co-located with the care model operational transformation team.
Working informally with system stakeholders, also served to develop a network of links and relationships that in turn led to increased access, as confidence and trust in the RiRs grew..The impact on access was further increased when the RiRs followed guidance from senior NHS managers to 'go where the energy is' (Meeting note 005, 23.02.16) and identify champions and receptive teams or settings in which they could colocate.
In the case of the WBC programme, co-locating one RiR with a partner voluntary sector organisation in the case study locality facilitated in-depth qualitative case studies and practitioner observations.Use of this evidence further increased access and impact.After five months of investing time and capacity in building relations and networks, one RiR observed following a locality 'Redesign Implementation Meeting', how the chair applied the plural, personal pronoun 'we' to express that RiRs were part of the integrated team (Observation Note 021, 27.07.16):

Vertical and horizontal integration
The connections and 'links' arising from being embedded in local teams and networks enabled the RiRs' findings to be 'fed back at various forums' and promote the 'work of the teams' (stakeholder survey), vertically within the ICO and horizontally between stakeholders.In this way, the RiRs not only facilitated the flow of information and evidence but -from a position of independence and therefore legitimacy -also provided 'voice' to and validation of frontline teams' work, which in turn could impact on staff morale and their engagement in service transformation.An example of this was co-presenting the locality case study together with all professional representatives of the Multidisciplinary Team in a plenary and panel discussion at a regional conference (University Clinical Schools 4th Annual Conference,16.05.18).This further enhanced interest in research and using it to feed forward and drive improvement.As one Locality Operational Manager noted: Inspiring my management team -opening their eyes from their immediate business, giving them space to think/reflect review.Real value to my leads and teams to have their voices heard and made to feel they add value and influence the development of services

Community engagement
As the RiRs' networks grew, this in turn mediated further horizontal engagement with other partners in the system, including social care, the voluntary sector, and patient and public representatives.Relationships were strengthened by the RiRs undertaking a range of engagement activities, including a series of co-design workshops on multi-disciplinary team working, attending and presenting at Patient Participations Groups (PPGs) in primary care, and ICO public engagement meetings.They also supported the Clinical Commissioning Group, the Carers Service and HealthWatch to design and, in some cases, co-produced public consultation surveys.The RiRs also held two qualitative interviewing and analysis workshops for the Carers Service 'Carer evaluators' and HealthWatch volunteers.This not only served to increase capacity and capability in the wider system but also increased the relevance and validity of co-produced insights, whilst raising awareness of the new model of care and its impact.As one Carers service manager commented: Hopefully some of the evaluations that Carers Services have completed and shared with them have helped to develop their overall understanding of the ICO from a Carer's perspective It also facilitated widening trusting relations between the voluntary sector and the ICO, opening up space for closer working on the WBC programme and focusing on patient experience to drive changes.

Learning and improvement
Learning and improvement cycles describe a process of mutual learning:  Learning is a key part of improvement: it tends to happen first and improvement can follow. Learning is reported to be more pronounced at individual (rather than organisational) level through direct contact with RiRs.
 Organisational learning takes time to filter through individual learning, especially when coupled with evaluation findings (a latent process).
 Learning can re-affirm assumptions leading into developing innovations or a more radical rethink when assumptions on expectations are not met.
 Likeliness of emergent improvement can be enhanced through interaction with access and expectations cycles.
 Concrete improvement outcomes are hard to attribute directly to RiR activities.
The RiR activities were recognised by survey respondents, in that RiRs:  provided additional resource (n=21 of 376 codings); not doing things for people but co-producing with them (see expectation management);  gave ad-hoc advice (n=20; which increases access, see above);  provided different perspectives or challenge (n=25); which may have resulted in more or less direct learning, and hence relates to potential of improvement or disengagement;  brokered knowledge and intelligence and made links horizontally and vertically across the system while mobilising and connecting champions (n=23; see access, which may lead to improvements via the stakeholders engaged).

RiR attributes
Stakeholders provided numerous responses to the qualitative questions on the attributes required of an RiR, as well as perceived challenges and improvements.RiR learning and specific responses are summarised in a job profile in appendix 2. Overall ranking the thematic content through the prevalence of codings, the following RiR job profile emerged: communication (n=28) and relationship skills (n=13), experience (n=30), training (n=12), knowledge (n=5); analytical (n=12) and judgemental (n=10) skills; mental effort (n=9); emotional effort (n=7); responsibility for improvement (n=4); and responsibility for ethics (n=3).
Many respondents noted that the mix of skills and behaviours required was beyond that they considered typical of an academic (e.g.academic training (9); knowledge of wider evidence (2)), in particular around aspects of communication (e.g.Ability to engage with diverse stakeholders (10); Ability to communicate with non-researchers ( 8)).In addition, the detailed themes around experience could be separated into general experience that was required (n=22), and specific experience to be acquired while being embedded locally (n=10).Stakeholders therefore seemed to emphasise a mix of formal knowledge mobilisation as well as experiential knowledge of the systems to increase relevance and potential impact of the role.Tailoring these skills and behaviours sensitively to different people and contexts was widely recognised as important to building relationships, as expressed by this senior manager illustrating the confluence of attributes listed above: A good ability to connect with all personnel, to understand their roles and how to harness the information that they require so that they can do their job better.A good communicator, good energy, a belief in what they are doing, a broad understanding of the environment that they are called to research in and the problems that it is facing, so better equipped to engage and deliver relevant research.

More direct learning and improvement
In the stakeholder survey, evidence use was ranked as the area in which the RiRs had had the greatest impact, 71% of respondents recognising some, moderate or a great impact (see impact domain results in Figure 1 above).This was supported by qualitative comments from the survey which suggest that implementing services 'at pace', finding dedicated analytical capacity, with ever-changing cost-saving and performance imperatives was hard.Stakeholders appeared to have scarce resource to reflect about what the service changes were designed to achieve and were working as desired, focusing on "transactional" rather than "transformational" change (Observation note 050, Care Model Delivery Group; 03.10.17).This might also have been related to timing, as the RiRs started during a busy care model implementation phase and 6months after the merger of the organisations.
The sense that the researchers added a perspective which was perceived to be independent and objective suggests that stakeholders value the external source of validation through (the sometimes symbolic) use of evidence, particularly when this aligned with their experiential insights.
Research has backed up with evidence some of my assumptions and understanding around how the organisation works and its culture.An external objective view which is evidence based, may have somewhat helped to influence the approach of the senior leadership of the Trust (Middle manager) were developed into recommendations in the 12-month report.All of these key findings were picked up to various degrees in programmes of improvement and implementation work in the subsequent months.
The explicit use of analytical and emotional material presented in a novel way was instrumental in facilitating stakeholders' understanding about the RiR model (aligning expectations) and its potential value to the organisation.This not only led to the ICO recommissioning the RiRs for two more years but also to work on the part of one of the Directors to develop the RiR model within other areas of the organisation (acute services).In terms of wider impact, RiRs and locality teams were co-hosting learning events for external visitors including NHS England, NHS Improvement, the International Foundation of Integrated Care, and academics (like those involved in social prescribing).The locality case study data also helped securing a nomination for the nationally renowned Health Services Journal Award 2018.

Discussion
Key Findings: This paper adds to the recently emerging case-based evidence base around a new approach to mobilising actionable knowledge: the applied, action-oriented, translational model of researchers-in-residence.
Our study reports evidence of impact on individual stakeholders, the organisation and to a lesser degree the wider system.We identified three domains in which feedback and feedforward loops could create virtuous or negative cycles or impacts.For the model to succeed in terms of facilitating learning and improvement, critical investment into awareness or active resolution of expectations, and promotion of presence and relationships in key settings to enhance access has to be made, with an on-going risk of disengagement.As trusting relationships are established, learning may be more likely to happen on the levels of individual capacity which may indirectly impact on organisational learning and actual service improvement.The above process is more straightforward when learning and co-produced output align with expectations, underlying models and assumptions, and tends to be more resource intensive and risky when challenging those.
RiR attributes and activities appear to be wide-ranging and beyond those that might traditionally be expected of an academic role.People, communication and negotiation skills are central to engagement, and critically feed into access and expectation cycles.Receptive context is also important in this and finding the energy and leaders open to working with RiRs.This is set against a background in which resources are scarce and there is pressure to implement change at pace.
The key enabler for the above to happen was the building and maintaining of trusting relationships, which took time.This was facilitated by listening and giving without expecting immediate return building general capacity for evaluation and research through ad-hoc consultation, while being intentionally open for this openness and generosity to yield returns in future.We found substantive co-location to be an important factor, as this allowed relationship, shared understanding, and joint ownership to emerge.

Strengths and weaknesses
This is the first study that explores the RiR model from the perspective of a wide range of stakeholders with whom they have engaged mainly around evaluation of existing service innovation rather than researcher-led service development.It not only assesses their perception of impact on them and their organisation but also their views on the key attributes required of the role when considering the embedded evaluation function.
We reduced the likelihood of reporting bias by conducting an anonymous online survey.
Although development of our theoretical model was driven largely by respondents' comments, the validity of the findings was increased through triangulation against our own field notes captured longitudinally during our research.
Nevertheless, this is one case study, focused on integrated care and comparison with other sites would have strengthened the validity of the findings.As the paper builds on a single case study and from the stakeholder perspective there is also an emphasis on learning and improvement for the organisation -rather than a sense of learning applicable beyond.The wider project always had an explicit dual aim of creating transferable knowledge, the outcome of which will be reported in future papers.
Another potential weakness relates to the embedded nature of the researchers, which puts RiRs at risk of 'going native', undermining their objectivity, i.e. when using NHS/University email and respective lanyards selectively.Although building trusting relationships can yield access and rich insight often not readily available to traditional researchers, it could also taint or compromise the researchers' interpretation of events trading off insider status with objective, external judgement.Depending on epistemological stance (Greenhalgh 2017) the 'in-betweenness' of the RiR model (Vindrola Padros et al 2017) can be seen as strength or a weakness, arguably increasing the risk of biased reporting.For example, in our case several evaluations of terminated service pilots we conducted were only reported internally.In addition, the participatory, action-orientated nature of the work means researchers inevitably have a stake in the outcome.If their future funding comes from the host organisation, there may also be a conflict of interests when reporting findings.
In this study, bias might have arisen, for example, through the choice of service innovations the RiRs were eventually tasked to evaluate, opting for projects with more perceived engagement and leadership.To mitigate this risk of bias , we tried to ensure rigour through having two RiRs with complementary skill sets (to enhance triangulation), as well as regular meetings off and on site with external academic supervisors to provide critical, objective feedback and oversight of the work, a model reported helpful elsewhere (Vindrola Padros et al 2018).

How the key findings relate to other relevant studies:
The impacts of the RiR models, as reported here concurred with 'expected impacts' noted by (Vindrola Padros et al 2018)Our findings also resonate with other RiR case studies that noted the need to identify and work with 'champions' and 'allies' as an important strategy for gaining entry and engaging staff (Vindrola Padros et al 2018).In our case the close working relationship with the central and critical QI champion [a coauthor], and through gradually enticing senior managers, aligns closely with learning from other participatory QI initiatives (Lalani et al 2018).Beyond what existing evidence says about the importance of senior champions, we see middle managers (who were biggest group of survey participants) as critical catalysts for indirect impact strategically and operationally.In line with existing case studies of co-produced work (Heaton et al 2016), we would also add that clinical and frontline staff are key enablers ('active agents') for change.As with existing accounts from other RiRs (Eyre et al 2017, Lalani et al 2018), our case study suggest that this can be a resource-intensive, compromising and demanding task.The pace and timing of informal decision making also reduces the scope to seek and use evidence to influence decisions (Wilson et al 2017), something that came out clearly as a theme in our study.

Policy, practice or research implications and/or recommendations
 Protocol Flexibility/Governance: Recommendations from other embedded studies (Vindrola Padros et al 2018) suggest that terms of engagement should be agreed from the outset in order to manage expectations, and that regular iterations of feedback with teams and management groups should be scheduled.With regards to the former and in our context of service evaluation, we would add that absolute certainty in terms of engagement is impossible and even unhelpful with contexts changing.However, regular meetings are essential.Indeed, the RiRs felt most impactful when being part of the team, and formally embedded in existing governance arrangements , rather than adjunct from it .
 Timing and boundaries: balance to be struck between serendipity, opportunism/flexibility, intention and iterative review of strategy to determine where to create the most impact (linked to expectations, access).
 Flexible and matched funding to enhance capacity: Our project was core funded by a local charity which allowed some flexibility in the protocol.NIHR -CLAHRC funding also contributed to the time of senior supervisors, something that might be extended to roll-out the programme across geographic footprints.Developing a sustainable source of evaluation capacity (i.e. through an ongoing consultation service) across partners could serve as a pipeline for research collaboration.
 Length of projects: We were also maximising the length of input and potential impact through parttime working (1.5FTE between RiRs) which in turn allowed time for other projects and co-bidding for additional funds.
 Co-location and honorary status: We also think that it is necessary to be completely and sustainably immersed, and that the RiR contracts specify that the place of work is on site, holding honorary contracts to enhance insider status and data protection.
 Job profile: See appendix 2 as seen from a stakeholder perspective.For the purpose of service evaluation and applied research, we would see the RiR as an academic post, rather than an NHS post .This is crucial to the impartiality of the role.Middle to senior researchers are probably required for the post to assure they are autonomous enough.We are looking to a current national study which seeks to address the gap in transferable learning and to develop practical guidelines and role specifications (Marshall et al 2018).
 NHS training and induction: job profile shows how important it is for impactful evaluation to bring and acquire experience of the health and social care systems (not necessarily clinical), something described as a 'culture shock' by other embedded researchers (Vindrola Padros et al 2018).We would strongly endorse recommendations by RiR colleagues (Vindrola Padros et al 2018) that suggest a clear assessment of the set-up before allowing a minimum of three months of an introductory period.In our case this took 6-12 months.
 (Peer-) Support: Having two RiRs with complementary skill sets, peer support, and autonomy, supported by respective clinical and policy Principal Investigators as well as local champions was crucial in both maintaining academic professional identity and integrity and sustaining the role  Joint ownership: We would add that regular iterations of feedback and sense-checking is crucial (our RiRs did on average two presentations of emerging findings per month), in the sense of ensuring that there is actual co-production in the process of refining and enhancing insights and their potential impact.

Conclusion:
Co-production of actionable knowledge appears to be real and relevant in this case study on the evaluation of integrated care model innovations, where the evidence base is inconclusive and where interventions are complex.Theory is emerging and needed to address the challenge around a causal understanding and attribution in a rapidly changing policy and practice context.The RiR model is one way to create relevant and rigorous evidence and to create ownership for change.In this, the RiR model could be considered a complex intervention in its own right.It appears that the same partnership principle driving integrated care, the pooling of resources for mutual benefit, hereby extends to researchers fulfilling an important improvement function as part of the wider multidisciplinary team.
However, although creating a legacy of independent evaluation without on-going academic input was a key aim and starting point, our evidence is inconclusive of whether this might be sustained without continued presence of RiRs in the system.There is also the question as to whether this example is just a way of creating a 'learning organisation'.The expectations from both academia and practice seems critical in this.In our case, expectation around summative evaluation clashed with RiRs' normative starting point that formative and summative are complementary, that implementation and effectiveness are inextricably linked, and that an openness to what 'doesn't work' is critical for learning and actual improvement.Yet this takes time, resources and mutual commitment to reach a plateau (less investment into expectation and access), from which both academically rigorous output and long-term and more tangible improvement in practice can be created.

________________________________________________________________________________ 1 .
Overall, what difference have the researchers-in-residence made to your work?
, following the development of the 'Mrs Smith model' and the creation of the Torbay Care Trust in 2005 (Thistlethwaite 2011).The ICO's integrated model of care incorporates over 30 service innovations and enabling functions that cover prevention, voluntary sector social prescribing, selfcare, proactive transition management and intermediate care.Against this background, strong emphasis has been placed on the need for research and evaluation to evidence and support desired changes.Torbay Medical Research Fund, the ICO and the The National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) South West Peninsula invested in research and evaluation through the Researcher in Residence model over a three-year period from April 2016-2019 initially.The two RiRs were employed and supervised by the University of Plymouth and based within the Quality Improvement team for reporting, support and development purposes.The two RiRs were provided with honorary contracts which enabled them full access to all services and data.The RiRs based themselves in various locations across the ICO footprint to enable them to work alongside teams.

Figure 1 :
Figure 1: Assessment of RiR impact across specific domains (stakeholder survey)

Reflecting back, what impact have the Researchers-in-Residence had on your work and your thinking in developing the care model?
Feel free to specify how giving examples, or proceed to specific impacts below: 2. 4 Thinking organisationally what impact have the Researchers-in-Residence had on your work and your thinking in developing the care model? Influencing operations and planning (MDTs, care delivery etc.) 3. How has the Researcher-in-Residence model worked best in relation to you, your team or organisation?Please give examples and explain how 4. In what

ways could their engagement with you, your team or organisation be improved?
Examples include how we communicate the role and what we are trying to do, access to the researchers in residence, the timing of engagement etc. 5.

From your perspective, what attributes does a Researcher-in-Residence need (e.g. skills, knowledge, experience, background) 6. Please indicate your role (optional)
Appendix 2: RiR Attrributes Overview table with content analysis and prevalence of codings from N=41 individual stakeholder responses to the impact survey questions: "From your perspective, what attributes does aResearcher-in-Residence need (e.g.skills, knowledge, experience, background)?"and: "In what ways could their engagement with you, your team or organisation be improved?Examples include how we communicate the role and what we are trying to do, access to the Researchers-in-Residence, the timing of engagement etc." Codings were populated across a relevant template content drawn from the generic National Job Profiles for an NHS "Healthcare Science Practitioners" (http://www.nhsemployers.org/yourworkforce/pay-and-reward/job-evaluation/national-job-profiles/health-science-sevices).Provides highly specialist advice and, or training to own and other professions in specialist area of activity; may undertake research and/or development in specialist field 3. Supervises and/or trains less experienced staff/students/trainees; may lead team for own work area Communication (28 codings):Effective communicator (7); Ability to engage with diverse stakeholders (5)/staff (3)/community (2); Ability to communicate with nonresearchers (6); Listening skills (3); Accessible writing (1), Presenting in engaging format (1)

2. Knowledge, Training & Experience Highly specialist expertise underpinned by theoretical knowledge & experience General experience required (22 codings
): Understanding health and social care systems (8); Method Mix (6); Academic experience (6); Knowledge mobilisation (2) "Understanding of operational pressures NHS and social care"; "NHS knowledge and experience is crucial in this role"; "Understanding of current and past strategies and what has worked/hasn't worked and why"; "Experience in several research and evaluation methods.";"wide experience of the types of research methods"; "Experience of completing research in a clinical setting with people who have complex medical needs"; Scientific background around what is known in these fields already"; "Knowledge of research -which can be practically applied"

. Analytical & Judgemental Skills Judgements involving complex facts or situations, requiring analysis, interpretation, comparison of options Analytical (12 codings):Openness
Training (12 codings): Academic background/expertise (9); "All the knowledge and skills you would expect of a Researcher"; "An academic Final manuscript(NOT anonymised)

To make people comfortable so can talk openly" 14. Mental Effort Frequent requirement for concentration, work pattern unpredictable; occasional/frequent requirement for prolonged concentration
Individual and organisational consent was obtained, while ethical approval was granted through 'Proportionate Review' by the NHS Health Research Authority (Research Ethics Committee reference: 17/LO/1745; Protocol number: PSMD-208147-SA-FG-034; Integrated Research Application System project ID: 208147).
Mismatched expectations of research and evaluation and what it could deliver was identified in the RiR reflective notes in the first year, in particular.The senior leadership wanted to know whether the care model interventions were working.There was little recognition at the beginning that the care model would take several years before detailed outcomes could be measured.It was a challenge to manage expectations as illustrated by the RIR reflections captured after a meeting of the research team with executives discussing findings of our 12-mth report (Observation note 047, 21.06.17): short: difficult but amiable enough meeting as (again) mostly about managing expectations about evaluation and what might be research going forward.It was important for building trusting relationships, which is a key message we emphasised (as well as our role in driving culture change).Didn't go into detail of findings and not sure if they read it, did not agree concrete action about further dissemination yet.Most interested in whether 'it' [the care model] 'works' and overall disappointment that our report does not deliver this.