Introduction

Nature-based solutions (NBS) are increasingly recognised as an effective response to a number of major urban challenges. These include heatwaves1,2,3, flooding4,5,6, water quality7,8 and public health and wellbeing9,10,11. While the concept of NBS emerged as recently as 201512, the idea of using urban nature to address these issues also features prominently in the more established fields of ecosystem services (which emerged in 2005)13 and green infrastructure (2002)14.

Despite mounting evidence of their benefits, strategies built around NBS are seldom practically realised15; implementation in cities has been slow, inconsistent and often limited to demonstration sites16,17,18,19,20,21. For example, 6 years after Copenhagen embraced green infrastructure as a response to its acute flooding problems in 2011, the implementation of green infrastructure was only just ‘taking off’ in 2017, and remained highly contested22. Even retaining existing urban NBS remains a challenge; tree canopy cover, central to mitigation of urban heat island effects, is declining in many cities23,24. For example, metropolitan Melbourne experienced a loss of 2000 hectares between 2014 and 201825. In the US, an average of 36 million trees were lost each year from urban areas between 2009 and 201426.

Barriers within the organisations responsible for implementing NBS are frequently identified as a primary reason for limited NBS delivery16,17,23,27,28,29. Delivery organisations have significant path dependencies; existing regimes are self-enforcing, and change is difficult30,31.

The reasons for non-delivery have been characterised in detail in a broad range of literature. Barriers are highlighted in studies focused on urban forestry32,33, urban water management18,29,34, nature-based solutions35,36 and climate adaptation37,38. The issue has been investigated through the lenses of mainstreaming39,40, governance19,21,33,34,41,42, transitions31,36,43,44,45 and general analyses of barriers17,23,46,47. Papers describing the barriers to NBS have drawn on interviews with experts17,23,33,34,36,48, and direct project experiences49,50. At the time of writing, we are aware of nine review papers that present typologies of barriers to NBS delivery, based on systematic reviews of the considerable literature16,18,21,29,32,37,41,46,47.

These studies have identified a largely consistent set of eight essential (and frequently lacking) traits for successful NBS implementation in local government. Leadership support is critical, both at the political and executive level18,20,21,29,34,41,51,52. A project team with the right capacity and timeframes to implement projects is also important32,37,51,53,54, as is a framework of internal mechanisms that facilitate the delivery of NBS, including clear approval processes, supportive policies and laws, and well-established standards for NBS design and maintenance16,18,20,35,47,55,56,57. A positive, supportive organisational culture for delivering new projects is also necessary, recognising that new NBS projects often have inherent (and novel) risks and trade-offs17,19,20,32,33,47,52,58. Finally, access to teams within the organisation that are both suitably skilled and supportive is vital16,18,23,46,54,59,60. Beyond the organisation itself, it is common for other levels of government to play an important role in approving aspects of NBS projects; an absence of support or clear process from higher regulatory authorities can pose a significant barrier32,33,38,44,51. Effective community engagement is also noted as important, recognising that many NBS need public support and/or private property owner consent to be successful23,37,38,46,49,61,62,63,64.

While the barriers to NBS delivery have been the subject of significant attention, the implementation gap persists, with recent publications continuing to note the difficulty of NBS delivery33,36,52,57.

A range of theoretical frameworks offer insight into how the implementation gap might be addressed. In the field of governance, The Policy Arrangement Model65 has been used to conceptualise governance in urban forestry32,33 and urban stormwater management21,34 as the temporary balance of a set of actors, discourses, rules and resources; changes to these variables may lead to changes in governance.

In the Policy Arrangement Model, each of these four elements is significant, as is their interplay33. The actors included (or excluded) in a policy arrangement are crucial, given the range of agendas in typical stakeholders (e.g. politicians, community groups, chambers of commerce, financiers etc.), as are the relations between these actors (some may operate as coalitions, or as antagonists). Discourses include tacit and explicit conceptualisations of what the policy problem is, how it should be solved, and what values matter most. These are important in lending legitimacy to rules, which define interactions and roles between actors. These may be as formal as laws and design standards, or as informal as a set of undocumented organisational processes and norms (e.g. “talk to Anne in our compliance branch, she usually decides what is safe”). These elements are all vital in determining who deploys resources such as staff time, skills, budgets or equipment, and how they are deployed. Collectively, the dynamics between these four elements constitute a policy arrangement; changes in one element have the potential to affect others, and in turn spark shifts in governance65. However, these systems can be strongly entrenched66. Governance shifts are theorised to be typically driven by at least four factors: policy entrepreneurs (or ‘champions’), shock events, socio-political changes and ‘adjacent arrangements’ (developments in policy domains in related sectors or institutions)67.

Policy entrepreneurs are also a focus of mainstreaming research, which highlights how these individuals advance NBS uptake by working within organisations to involve key stakeholders, engage citizens and contract technical expertise while incrementally introducing NBS considerations into planning practice35. This work is conceptualised as ‘horizontal’ mainstreaming, as officers champion NBS across their organisations, but it is argued that this must be supported by ‘vertical’ actions by top-down actors (such as executives and elected leaders) with the power to determine resource allocations and organisational structures39,40.

To support NBS development and planning, the European Union’s Horizon 2020 programme initiated a series of large international demonstration projects, each involving collaborations between a number of cities, consultancies and universities. These include the UnaLab, ProGIreg, Connecting Nature, GrowGreen, Urban GreenUP and EdiCitNet projects68. These projects generally fund dedicated staff, as well as on-ground delivery of NBS, and have potential to address some or all of the barriers to NBS delivery that cities face. When considered in terms of the Policy Arrangement Model, the new actors, discourses and resources introduced by these projects all challenge the ‘temporary balance’ theorised to constitute the organisational status quo65. These projects also may encourage governance shifts67, by both facilitating the hiring of NBS policy entrepreneurs, and increasing a city’s exposure to influential adjacent arrangements in other centres of NBS expertise, such as university research units or exemplar municipalities. With the involvement of organisational champions/policy entrepreneurs, mainstreaming activities such as stakeholder outreach, citizen engagement and intra-organisational collaboration become increasingly possible35.

This paper investigates the Horizon 2020 NBS project, Urban GreenUP. Urban GreenUP focuses on supporting partner cities to prepare NBS plans, as well as funding a multi-million Euro programme of investment in NBS interventions including floating vegetated islands, green walls on private structures, and streambank renaturalisation. The seven cities participating actively as project partners are Liverpool (UK), Ludwigsburg (Germany), Mantova (Italy), Valladolid (Spain), Izmir (Turkey), Quy Nhon (Vietnam) and Medellín (Colombia). This group of cities represents a wide range of governance arrangements and urban contexts in which NBS delivery occurs; Liverpool is a significant post-industrial centre emerging from sustained economic challenges compounded by government austerity, while Mantova has large areas of UNESCO world heritage and a legacy of industrial pollution. Quy Nhon is a coastal holiday town fairly new to NBS, while Ludwigsburg has extensive environmental legislation and has already successfully carried out major streambank restoration works on their local river. The former is largely governed by provincial government, with more operational management at a local level, whereas the latter has individual portfolio mayors, including one for the city’s environment. Valladolid has a population of 300,000 and a fairly compact urban form, whereas Medellin numbers over two million residents. We were able to work with each of these cities, effectively capturing the full range of capabilities and experiences in the Urban GreenUP project, and a significant variety of landscapes and organisational contexts in which NBS may be implemented.

While the ‘generalisability’ of case studies is often limited, the constraints can be at least partially addressed through strategic sampling of cases69. Our sample is diverse, and while limited to seven cities, it does represent the full cohort of cities participating in this major EU programme designed to promote NBS innovation. A smaller sample size allows for a close, qualitative study of each case. Urban GreenUP presents a valuable opportunity to investigate the persistence of NBS barriers within local governments with ambitions for NBS implementation, with implications both for future innovation-oriented programmes such as Horizon 2020, and potentially the broader practice of NBS delivery in cities.

Many cities beyond the Urban GreenUP group are preparing new NBS plans and programmes, and could benefit from insights arising from this study. Each GreenUP city is embarking on NBS planning and delivery and, at the time of our research, each had an individual or team employed with a specific NBS delivery role (with potential to serve the policy entrepreneur role highlighted in the literature). Local government often plays a key role in the implementation of urban NBS39,47,70, and Urban GreenUP places these organisations at its centre. Citizen engagement is an explicit focus of the project, as is the making of plans; these are both emphasised as opportunities for mainstreaming new practices35,71.

We analyse the NBS implementation capacity of the cities within this study using an approach generally consistent with the practice of theory-based evaluation72,73. This ‘theory-based’ method breaks an implementation programme into its component elements, and assesses each element against available theory regarding what is required for success. This poses significant advantages over other evaluative methods because it focuses on the causative elements that lead to policy success or failure, rather than just the final outcomes achieved74, and is therefore more conducive to reforms of the institutional barriers discussed above.

Theory-based evaluation typically takes place at the end of projects, but ours is ex ante; an approach noted by Weiss in her seminal outline of theory-based evaluation as having the potential to improve programme planning72. Evaluative practices have been noted as a particular weakness in local government NBS programmes, both at the political and officer level, due to a fear that acknowledging problems would lead to criticism of failures36. We sought to mitigate this issue both through use of an ex ante approach, and by designing a tool that creates distance between evaluators and individual practitioners.

This paper investigates the enduring difficulties faced by cities seeking to deploy NBS, in the context of a major NBS project spanning seven cities. We had two key research questions. First, do case study cities have the capabilities required for successful NBS delivery, or are there barriers that continue to make this difficult? Second, does identifying and measuring a city’s NBS delivery capabilities facilitate improvements in these capabilities?

We elicited organisational barriers from NBS practitioners in participating cities using a purpose-built diagnostic tool. We used this approach because tools that enable organisations to learn about their success factors have the potential to address implementation gaps44,71.

The tool was developed to bring lessons from the literature into an operational context, by enabling practitioners to assess and rate their organisation’s capability levels across eight key areas, such as political support, alignment between teams, and technical knowledge (refer to Table 1). The tool posed a series of questions pertaining to each of these eight capability areas; users answered by selecting from a set of pre-defined answers. The tool associated each answer with a level of capability, which was reported as a final assessment of an organisation’s capabilities. This was provided to the practitioner directly on completion of the tool’s questions, enabling an immediate estimate of their organisation’s NBS delivery capacity, and a diagnosis of any key barriers they would be likely to face in future NBS projects. These results formed the basis for our reflections on NBS delivery capacity within Urban GreenUP, as well as discussions with practitioners to understand how results of the tool were received.

Table 1 Eight success factors for urban NBS in local government.

Our research proceeded in three steps. First, we drew on the literature to define a set of eight key capabilities—which we call ‘success factors’—for NBS; these formed the basis of the tool. Second, practitioners within NBS teams in participating cities used the tool to identify their capability levels. Finally, we interviewed users to evaluate the impact of the tool.

Results

City capabilities

The tool assesses eight broad capability areas for NBS delivery. These are grouped according to the barriers that were common across multiple disciplines (such as urban forestry and integrated water management) and framings of the problem, especially in review papers that outlined typologies of barriers16,18,21,29,32,37,46,47, and are detailed in Table 1.

Most of the participating cities faced deficits in multiple success factors, at a level ranked by the tool as either very challenging or critically challenging (Fig. 1). Furthermore, five cities have results rated challenging in at least half of the eight success factors. Stable political/executive support is a challenge in five cities. Four cities have unsuitable internal processes, strategies, regulations and/or policies. Notably, almost every city reported shortages of staffing, and for the majority of teams it appeared to be a serious issue. Challenges were noted in organisational culture in four cities, as were difficulties with other government departments in four cities. Factors considered relative strengths across the group included technical capability, community engagement and supportive internal departments. Two cities (City 3 and City 6) reported strengths in almost all capability areas.

Fig. 1: Summary results for each city.
figure 1

The full set of questions the tool uses to produce these results are outlined in Supplementary Table 1.

Reviewing individual responses to the questions within each success factor revealed more specific strengths and weaknesses (Fig. 2).

Fig. 2: Results across all case study cities and success factors.
figure 2

Highlighted questions were of particular interest; Q2.1 shows that very few cities had clear process for NBS implementation, Q3.1 showed that staff resources were usually very scarce, and Q6.2 indicates that when new initiatives fail or are difficult, a ‘never again’ response is common.

NBS approval processes were a key issue among the grouped issues of ‘processes/standards/policy/regulation’. Four cities do not have clear processes for NBS approval, meaning this must be negotiated case-by-case with other parts of the organisation (Fig. 2, question 2.1).

Staffing shortages were a consistent cause of low ratings in the ‘adequate and empowered staffing’ success factor (Fig. 2, question 3.1). Culture in relation to risk was also highlighted as a key barrier for most cities. When asked, “When a new initiative fails or is difficult, how do leaders and executives tend to respond?”, four users answered, “That was worth a try but let’s never do it again” (Supplementary Table 1; question 6.2). This response was rated ‘challenging’ by the tool but this may prove to be a more severe issue in cities establishing new programmes within risk-averse cultures.

A few nuances not highlighted in Fig. 2 are also notable. Low ratings for the first success factor ‘Stable executive and political support’ were reflective of a wide range of reasons—some cities lacked executive support (1.1), some, political support (1.2) and some, stability (1.3); there was no acute cross-cutting issue. Most cities reported having good, but general, policy and strategic support for NBS already in place (Q2.4). General support is not as effective as a detailed implementation framework, but it is also clear that policy is at least not an active source of obstruction.

The comparatively good ratings for the ‘Alignment of Internal Departments’ success factor was indicative of cities reporting that their engineering, design or maintenance departments “have basic knowledge and are tentatively supportive if risks are managed well” (Q5.1–5.3). While not obstructive, this tentative support is perhaps a less encouraging result than the rating suggests.

While the reasons for low overall success factor ratings varied, the tool revealed three consistently problematic capability areas: (1) acute shortages in staffing; (2) strongly risk-averse culture; and (3) obstructive or unsuitable processes for NBS approval.

Impact of the tool

Users generally reported that the tool accurately characterised their success factors, and considered the tool useful.

“I definitely agree with the success factors, I liked this very much. The multiple-choice options you provide are quite accurate, it’s quite easy to answer and reflects the differences between answers well. The answer options are well selected, the results accurately reflect my answers. When I saw the results I felt reflective about that – what I see now in the screen, Advanced Community Engagement Skills may be a problem for us. I know this is true, this part of the thing works quite well.”

Team member, City 5

“Yes, I think this is very good to make it more clear for us. We have it in mind, okay, this might be the problem but you collect the answers and then you see yes, this is difficult for us and how can we maybe solve the problem. So yes, I like this overview because it’s very clear. It covered the main factors for us.”

Team member, City 1

While the tool’s findings were endorsed, practitioners who used the tool already clearly had tacit knowledge of their own missing success factors. Users found it useful to have their strengths and weaknesses made explicit; however, the tool did not encourage new actions (such as organisational reforms), because knowledge was not what was preventing improvements to success factors.

“We agree. We know these are the barriers. (…) The problem is transferring between theory and reality (…) Our motivation is great, but it does not depend on us, the staff – we have an organisation, they decide who must work on this thing. This is a problem that we cannot solve alone (…) At the moment we cannot change it.”

Team member, City 2

A number of interviewees noted that if other teams were asked about the success factors, their answers might differ. In one city, users felt they could only use success factor ratings if they were produced through consultation with all the relevant internal teams.

Despite these limitations, users noted other potential strengths of the tool, including that it would be useful as a means to guide stakeholder deliberations, as part of the planning process within organisations to build alignment in early project phases, and in building awareness in decision-makers around the organisational requirements to execute projects effectively.

Interviews also shed light on the two very positive responses in a way that underlines the tool’s limitations. One team appeared uncomfortable with the tool’s probing questions, and another was newly-formed when the tool was offered to them. This highlights the tool’s dependence on informed and willing user input.

Discussion

Organisational barriers continue to limit NBS delivery, despite the popularity of the concept and years of scholarship identifying the barriers through a range of theoretical lenses. To interrogate the persistence of this problem in a practical context, we provided a tool for local government practitioners to identify success factors in their organisations. We sought to diagnose problems and encourage actions to build delivery capability.

We found that the cities embarking on NBS delivery as part of Urban GreenUP had strong capabilities in a number of areas typically identified in the literature as barriers. These included good access to technical skills and openness to citizen involvement. General (albeit tentative) support from teams across the organisation was common, as was a broadly supportive policy environment. This represents an important difference between our studied organisations, and the literature to date; project champions in Urban GreenUP are overcoming some typical barriers.

Three major enduring barriers were reported by most of the cities in this study. First, we identified a lack of clear organisational processes by which NBS are delivered. The absence of process creates a requirement for ad hoc negotiation within and between organisations to deliver NBS. This kind of negotiation is typically with teams that have other priorities, such as traffic engineering or park maintenance. While these areas are not necessarily actively opposed to NBS, the absence of process has been noted in the literature to make NBS projects more time-intensive and uncertain17,23,75. This is especially difficult given that the second barrier we identified was an acute shortage of staffing to manage NBS delivery. The third barrier we noted was that cultures around risk were cautious and punitive, in a way that has been emphasised in the literature as a barrier to innovation30.

Revisiting the Policy Arrangement Model, we may conclude that actors and discourses are shifting, but the intertia is primarily in the allocation of resources, and the ‘rules of the game’ (both in the tacit rules about risk, and more explicit rules about NBS approval). However, the findings of this study are especially interesting when considered in terms of the dichotomy of vertical and horizontal mainstreaming39,40. Horizontal mainstreaming, typically carried out by policy entrepreneurs, includes the establishment of new activities such as strategies and pilots, collaborations across the organisation, technical skill development and engagement of the public. The teams we spoke to were generally strong in these areas, with policy entrepreneurs actively pursuing these activities, aided in part by funding and knowledge resources provided by Urban GreenUP (for example, stakeholder engagement advice and access to specialist engineering consultancies).

Vertical mainstreaming, typically a responsibility of executives, includes a set of top-down actions within the organisation. These include setting new organisational norms (for example in relation to risk), modification of organisational rules and working structures to facilitate delivery, and the allocation of resources to support delivery (for example, by hiring staff). These correspond closely to the weaker success factor results in our study; it is this area of mainstreaming activity that appears to be much less developed in the cities we studied.

The need for vertical mainstreaming activities was underlined by our interview findings. Practitioners considered the tool effective and useful in the way it explicitly identifies and measures success factors, but it was clear that the most pervasive barriers in the studied cities were already very familiar to practitioners. The diagnoses revealed through application of the tool did not encourage actions to improve success factors; these problems persist because not because they are unknown, but because addressing them was not within the authority of the practitioners we spoke to.

Returning to the question of what can be generalised, we note a few points in our case studies that could apply to wider practice. The contrast between case and theory has again validated the suite of issues that cities face in NBS delivery, with practitioners affirming the eight success factors as comprehensive and familiar. The accuracy of the tool suggests it has promise as an efficient mechanism for injecting theory-based evaluative practice into workplaces, particularly as a focus for deliberation. Perhaps most interestingly, the way that the most enduring barriers (Risk Aversion, Resourcing and Rules) correspond to typical executive responsibilities suggests that this actor group may need to play an expanded role in cities seeking to initiate NBS programmes. Certainly, in large innovation-oriented programmes such as Horizon 2020, a more explicit stream of actions for senior organisational leaders appears to be warranted, as only these actors have the power to reform the challenging organisational barriers that we have identified.

We note three key limitations of the tool used in this study. First, this is a very diverse set of cities, and our tool is based on a literature that may not have given due attention to non-western governance models. For example, in Vietnam, most implementation is carried out by a City People’s Committee, subordinate to the Provincial People’s Committee, while the locally-elected People’s Council plays a fairly limited supervisory role. Second, the tool’s reliance on user input exposes the tool to the typical limitations of self-assessment approaches. For example, while a willing and experienced practitioner with an interest in building capacity might use the tool honestly to receive a meaningful response, other scenarios (for example, one where the user perceives reputational risk or a chance to access additional resources) may incentivise ‘gaming’ of the tool. Future studies could account better for this risk by establishing more anonymous input conditions, and including stakeholders beyond core delivery teams. Finally, there is likely scope to refine the tool’s categorisation of responses; in particular, we note instances where very general support from policy or other teams was considered ‘functional’, where this may amount to simply a lack of obstruction rather than any real facilitation of NBS delivery.

Methods

To arrive at our findings, we employed three key methodological steps:

  1. 1.

    Defining a set of ‘success factors’ for urban NBS, based on a review of the academic literature, for use in the self-assessment tool;

  2. 2.

    Assessing capabilities in seven case study cities via the self-assessment tool; and

  3. 3.

    Reflecting on the value of the self-assessment tool through semi-structured interviews with practitioners.

Defining success factors

Given that we sought to carry out a theory-based evaluation, our first step was to assemble and synthesise existing theoretical understanding of the capabilities required for NBS delivery. To do this, we identified barriers to NBS delivery through a review of the academic literature. Key references in this field were retrieved using combinations of the following search terms in Elsevier ScienceDirect:

(Path dependence or institutional barrier or organisational barrier or transition or institutional capacity) AND (NBS or Green Infrastructure or Living Infrastructure or SUDS or WSUD or IWM or urban ecology or urban forest or green space)

From the results of this search, we eliminated results that did not pertain to both organisational barriers and urban greening, producing a total of 37 peer-reviewed articles, including nine review papers that each outlined a typology of barriers to the implementation of urban NBS programmes and strategies16,18,21,29,32,37,41,46,47.

Each reference was reviewed with a focus on identifying barriers to the implementation of urban NBS interventions. Following this, individual barriers were synthesised into a set of eight overarching success factors for urban NBS. Barriers were reframed as ‘success factors’ to assist when communicating with participating cities, in an attempt to promote uptake and optimism in tackling organisational challenges.

To measure capability in the eight success factors defined above, the tool was prepared in a spreadsheet. For each success factor, The tool poses three to five questions designed to evaluate competency and reveal critical issues. For each question, the tool offers a set of pre-defined responses. To facilitate comparison between cities, each question had a limited set of response options. The responses represented a range of capability, from very high to critically low.

Answers to individual questions were considered in the context of their potential impact on successful NBS implementation, and were categorised as ‘optimal’, ‘functional’, ‘challenging’, ‘very challenging’ and ‘critically challenging’ based on the literature review.

As this tool to rate success factors for NBS delivery is novel, our categorisation of each answer relied on our interpretation of the literature, as well as our experience as NBS practitioners. This necessitated a judgement under some uncertainty; the scoring and rating approaches used in decision models can be a topic of significant expert debate, even when such tools are already in active use76. To mitigate this we tested the tool with practitioners by providing an early draft before their final use, as well as including an interview question about whether users agreed with the tool’s rating of their capabilities, and included a field in which users could adjust the final ratings the tool had allocated. Our testing and user feedback affirmed the tool’s ratings, with only one user applying the override to slightly adjust the tool’s assessment, and general support of the tool’s conclusions reported in feedback sessions.

Assessing capabilities

This research reflects on and critically analyses success factors for NBS delivery, using the Urban GreenUP project as a case study. Urban GreenUp is a large European Union research project, funded under the Horizon 2020 programme77.

Urban GreenUP focuses on preparing ‘Renaturing Urban Plans’ for these cities and aims to more broadly demonstrate techniques for NBS planning. The Renaturing Urban Plans will guide the strategic rollout of NBS in these cities to tackle local challenges such as urban heat islands, flooding, air quality and urban renewal. NBS was a strong focus for the city teams involved in the project at the time of our research; this context was ideal for our case study of barriers to NBS delivery.

The tool was issued to project officers in the local government of each participating city. Users were free to fill the questions with whomever they wished (for example alone, with their teammates, or drawing in consulting and academic partners). Seven cities used the tool; each received a tabulated output that offered the following information:

  • The tool’s overall assessment of the city’s capability for each of the eight success factors (terms ranged from ‘this is a strength for your organisation’ to ‘this may be a serious problem’)

  • A list of specific challenges flagged within each success factor (e.g. if one’s maintenance department has nobody with the suitable skills, this would be flagged as a critical issue).

The provision of plain English assessments of capability and flagging critical issues was included to aid interpretation of results and encourage active responses.

Reflecting on the tool’s findings

In our follow-up interviews, we ascertained how each partner used the tool and considered its findings. To encourage transparency, respondents were advised that results were anonymous. Accordingly, results are presented in a non-identifiable format. This was in accordance with the RMIT University ethics approval that was granted for this research (approval number: CHEAN A 21953).

Six months after teams completed their self-assessments using the tool, we followed up to understand how the tool was used, and to determine whether it was useful to understand the team’s strengths and weaknesses. Follow-up interviews were semi-structured, with responses prompted by a series of guiding questions, and completed in person or via video conference. Interviews were conducted with a single member of the participating team. The questions posed to each participating team to prompt responses were as follows:

  • How was the tool completed (e.g. alone, or with other people)?

  • Did you agree with the tool’s assessment of your government’s capabilities?

  • Will you change anything about your planning approach following the tool’s identification of your strengths and weaknesses?

  • Are there further improvements you would recommend to the tool?

  • Will you use the tool again? How?

Conversations took ~30 min and were recorded and transcribed. A thematic analysis was applied, coding answers manually and then identifying and counting common responses across the seven sets of responses78. For example, we received responses that indicated the tool was completed either alone, in pairs, with a team, and with multiple teams. Each interviewee’s answers were coded into these categories to enable counts. These are included in Supplementary Table 2.