Introducing the revised framework for developing and evaluating complex interventions: A challenge and a resource for nursing research

This invited discussion paper highlights key updates in the MRC/NIHR's revised framework for the development and evaluation of complex nursing interventions and reflects on the implications for nursing research.


What this paper adds
• A brief summary of the key points made in the latest version of the MRC/NIHR Framework for Developing and Evaluating Complex Interventions.
• Examples specifically from nursing research, to illustrate the key points made in the updated Framework.

Background
Complex interventions are omnipresent in the health service, public health practice, and social policy, e.g., social security, education, transport.They can have important health consequences.To maximise benefit, interventions should be developed, evaluated, and implemented with appropriate consideration of the complexities of their design and their interactions with the contexts in which they are implemented.In October 2021 we published a Framework for Developing and Evaluating Complex Interventions.The framework supports researchers, decisionmakers, funders, and others to approach complex interventions appropriately.This was an update of Medical Research Council guidance originally published in 2000 (Campbell, 2000), and first updated in 2006 (Craig et al., 2006), and drew on various methodological and theoretical developments of the last 15 years.The Framework document itself is published in full in the NIHR Journals Library (Skivington et al., 2021a).A reprint of our article introducing the Framework (Skivington et al., 2021b) is provided in this issue of International Journal of Nursing Studies (IJNS) (Skivington et al., 2024).
In this short introduction we highlight four key points from the Framework, providing further examples relevant to nursing research.

The definition of complex interventions is updated, highlighting the relationship between the intervention and its context
We added to the previous definition of 'complex intervention', most notably by paying increased attention to 'context'.We state that complexity arises through characteristics of the intervention itself; and/or interactions between the intervention and its context.Context is any feature of the circumstances in which an intervention is conceived, developed, evaluated, and implemented; and is dynamic and multidimensional.Effects of an intervention may be highly context-dependent, and as such, an intervention developed and shown to be effective in one context, will not necessarily be effective in others.The Family Nurse Partnership is one examplea preventative home visiting intervention by specially trained family nurses, to reduce maltreatment, improve maternal and child health, and improve child developmental and educational outcomes.The intervention was shown to be effective in three trials in the USA (Olds et al., 1986(Olds et al., , 2002;;Kitzman, 1997).When implemented in England, however, a pragmatic randomised controlled trial found no benefit on the primary outcomes, and the authors concluded that the short-term evidence did not justify the continuation of the programme (Robling et al., 2016).Usual care in England includes free access to statutory health and social services, including obstetric antenatal care, and health visiting as routine.This is substantially different to that in the USA.These contextual differences may have played a role in the difference in intervention effectiveness.
We would argue that no intervention that involves human behaviour, agency and social relationships (e.g., interaction between someone delivering and receiving an intervention) is truly simple, and even seemingly simple interventions have varying effects in different contexts.
2. There are various important aspects of complex intervention research-'core elements'-that must always be given due focus We identified six 'core elements' of complex intervention research and suggest that these are revisited continuously throughout the research process, and particularly before moving to a new research phase (e.g., from development to feasibility).The full Framework document provides more detail on each of these core elements, and how they should be considered throughout the research phases.With specific reference to nursing examples, the core elements are: i.Consider context: as discussed above, complex interventions can be expected to vary across contexts in their effectiveness.Researchers should think carefully about how the intervention will interact with its context and which aspects of context should be taken into account throughout the research process from intervention design through to evaluation and implementation.ii.Develop, refine and (re)test programme theory: it is important to develop a programme theory to describe how an intervention is expected to lead to its effects and under what conditions.This is necessary for intervention developmentjust because something seems like a good idea, if there is little consideration of the mechanisms through which it could affect change, and of the potential unintended consequences, there is a risk that the intervention will not be successful and may even be harmful.For example, physical restraints, e.g., bedrails, belts in beds and chairs, chairs with fixed tables, are commonly used in hospital settings to prevent falls and injuries.However, studies have shown that such interventions are not effective at what they set out to do, and in fact can have adverse effects, e.g., decreased mobility, poorer wellbeing, increased feelings of discomfort, or no benefit (Abraham et al., 2020) (Petkovic et al., 2020).Building partnerships and collaboration across stakeholder groups, crucially including those with lived experience of the issue, and having shared goals and co-designed working protocols can be a key part of a successful intervention (Wong and Wong, 2020).A systematic approach to gather patient perspectives on their preferences could be used to design more patient-centred interventionsbut also to inform how you might go about doing the research, as done in Petherick et al.'s (2006) questionnaire study to explore patient perspectives on preferences for laval therapy.In terms of evaluation, and for the findings to be acted upon, the research team need to gather views on what 'useful evidence' looks like to different stakeholders.iv.Identify key uncertainties: There are various questions that could be answered at each phase of the research process.Research teams need to identify the key uncertainties by considering what is already known and what the intervention programme theory, research team, and stakeholders identify as being the most important to address.The research responds to uncertainties and leads to more evidence.As such, uncertainties will change as evidence accumulates, so should be reviewed and updated at each phase of the research process.We often prioritise research that has a greater probability of finding a certain answer, even though the question may be of less importance.In many areas of intervention research, it may be useful to give higher priority to evaluation that is sensitive to complexity.For example, exploring what worked, what did not work, for whom, how, why and in what circumstances, as done in a realist process evaluation to explore different types of implementation programmes for urinary continence care (Rycroft-Malone et al., 2018).v. Refine the intervention: this is the 'fine tuning' or making changes to the intervention once a preliminary version has been developed, but not necessarily fully evaluated.We want the optimal version of an intervention to be evaluated and implemented, and ongoing refinement can support this.For example, Witzig-Brändli et al. ( 2023) used a refinement loop process when developing a consulting guideline as one component of a nurse-led self-management intervention for people with Multiple Sclerosis.Ongoing refinements were made following each iteration of testing with different stakeholder groups.vi.Economic considerations: complex interventions are often costly.The resources needed to implement them have opportunity costs, i.e., the benefits that could have been gained from alternative uses of those resources.They often impose costs on, and generate benefits for, a range of populations or organisations.Systematic assessment of these requires economic evaluation, i.e., the comparative analysis of alternative courses of action in terms of both costs (resource use) and consequences (outcomes and effects).In complex interventions, these may occur across different sectors or levels, so it is important to define clearly the perspective adopted in an evaluation.For example, the economic evaluation of a national smoke-free prison policy in Scotland used the perspectives of the healthcare payer, prison service, people in custody and operational staff, and showed that implementation of a smoke-free prison policy is cost-effective in the short and long term (McMeekin et al., 2023).Key to economic evaluation is the identification, measurement and valuation of the resources and outcomes according to the perspective adopted.The Framework highlights how these processes can, and arguably should, be built into each stage of the process of evaluation planning, design, execution and implementation.Doing so helps with understanding the problem and shaping the design of future studies.Boyer et al.'s (2014) study explores the feasibility of an economic analysis of a treatment intervention for children with maltreatment-associated psychiatric problems, which was adapted from the USA to the United Kingdom.
It is one example of early engagement with economists to support the development of programme theory, and in turn the evaluation design.The scenario-based analysis provided evidence of feasibility for economic evaluation and gave recommendations for the outcome measures in the evaluation.
3. The usefulness of evidence should be the basis for determining research perspective, research questions, and methods The original framework, published in 2000 (Campbell, 2000), arose through a translation of linear-type frameworks used to guide drug development research, and adapted for evaluating more complex interventions in public health and health care.The update in 2006 was much less linear and moved from researcher developed interventions to consider interventions more broadly (Craig et al., 2006).Yet it remained within a paradigm where the fundamental question was 'does it work?'.Many of the most promising interventions do not get, or cannot be evaluated in this way, e.g., service and policy innovation, population level policies.Other interventions may be shown to be effective in a controlled trial, but then the findings are not always replicated, either because it is not implementable or encounters implementation failure, it is not transferable across contexts, or because wider system effects emerge.
An example of this is hourly rounding in hospitals (where nurses proactively check on patients every hour to assess their needs, provide basic care, and address any concerns or questions they may have), where positive outcomes such as patient satisfaction, reduced call light use, decreased falls and pressure ulcers, and improved nursing workflow, have been demonstrated (Ryan et al., 2019).However, key challenges to implementation show nurses can struggle to prioritise hourly rounding amidst their other responsibilities, and that staffing shortages and high patient acuity make it difficult to consistently round on patients every hour.Whilst an intervention may be effective in a controlled trial setting, its effectiveness in real-world implementation may be limited by factors such as staffing, workload, and organisational culture.
We argue that there have been many researcher-led 'effective interventions' that have achieved little real-world impact due to lack of consideration of implementation requirements and stakeholder insight.To support the choice of appropriate research questions, we introduce four research perspectives: efficacy, effectiveness, theory-based, and systems.The choice of perspective should be governed by the research questions you want to answer, and these should be determined by identifying the key uncertainties that exist.The Framework also aims to move away from pervasive hierarchical thinking in terms of research methods.There are numerous approaches, and these should be seen as versatile, but not universal; a 'toolkit' of methods where the most appropriate is/are selected for the identified research questions, horses for courses if you like.
Recently, it has been argued that there is a "paucity of rigorous experimental research in nursing" (Watson et al., 2023), and that a renewed focus on theory-based evaluation in nursing is required (Wallner et al., 2023).The pluralism that we suggest makes the Framework relevant to the range of approaches used and useful in nursing research, without favouring one over another (Dichter et al., 2023), and highlights the elements at the core of the research process, regardless of perspective taken.

Consider adaptation and 'identified interventions' as well as researcher-developed interventions
We do not always need a brand-new intervention: the framework gives due attention to intervention adaptation, i.e., taking an effective intervention and adapting it for a new context (see also, Moore et al. (2021), guidance for adapting interventions to new contexts).For example, there is evidence that brief psychological interventions are safe and effective for those with antenatal depressive symptoms; Bitew et al. (2022) selected a particular intervention developed in South Africa and adapted it, following an adaptation model, for an Ethiopian country and cultural context.
Evaluation is also important for 'identified interventions,' such as the introduction of a new Government policy, or a service development like the introduction of the Family Nurse Partnership in Scotland (Cavallaro et al., 2023).Attention to the core elements of our framework in each research phase is important here too.Even although the intervention has been implemented, drawing on understandings of multiple stakeholder perspectives to develop programme theory and identify key uncertainties can support the decision to evaluate, the choice of evaluation design, and interpretation of findings (Wackers et al., 2023).

Concluding remarks
The 2021 Framework has been cited over 2000 times and continues to be referenced in funding applications.Rather than simply citing the Framework, we encourage readers to engage with the checklist for its use (see Appendix 1), and give full consideration to the core elements at each phase of complex intervention research.Perhaps because of funding mechanisms or career expectations, we often prioritise research that has a greater probability of finding a certain answer, even where the question may be of less importance.It is essential to consider the translation of the research evidence into practice; what has greatest potential for impact?Will the evidence tell us something useful about implementation and the possibility of adaptation or scale up?We hope that the Framework and checklist will help to answer such questions and make decisions on the research to take forward.
. Appropriate development of programme theory can help to avoid interventions being developed inappropriately.Intervention programme theory can also be used to guide the evaluation by supporting the prioritisation of research questions, ensuring that the evaluation is closely aligned with the goals and underlying assumptions of the intervention, and appropriate outcomes are used.Review and synthesis of the evaluation of other relevant interventions can support the creation of a theory-driven explanation of the planned intervention, and the components required and conditions under which any future interventions are likely to be successful.This type of work can provide a theoretical platform for developing and evaluating future interventions, as done in work to improve care for care home resi-.Engage stakeholders: different people have different perspectives of a problem or issue, and it is important to consider these in making decisions about what the research questions are, or how they should be prioritised.For example, patients may view the 'same' situation quite differently to nurses, as shown inHarris et al.'s (2017)exploration of service users', carers' and professionals' perspectives and experiences of antipsychotic prescribing.If an intervention is developed from one perspective, particularly if that perspective is of the research team, then it holds less promise for successful and sustainable implementation Otis et al. (2023)017)ntia and faecal incontinence(Goodman et al., 2017).Within individual studies, updated programme theory postevaluation may therefore support future intervention development and evaluation elsewhere.Otis et al. (2023)describe how they have used programme theory to develop an evaluation of a 'new model of care for integrating children and young people's acute mental healthcare in a paediatric setting', and in turn how this evaluation will support further refinement of the initial programme theory.iii