A diagnostic tool for supporting policymaking on urban resilience

Urban resilience has become a popular notion among urban policymakers and scientists, as a way to deal with the many complex issues that cities face. While it has positive connotations and resonates with local urban agendas, it is not always clear what it means and what factors contribute to resilience. Additionally, critical literature observes that people's views on what resilience means can di ﬀ er strongly and the many choices that are made in planning and implementing resilience are often left implicit. In this paper, we describe a diagnostic tool that tackles these issues by (1) distilling resilience principles and narratives that provide a comprehensive picture of the di ﬀ erent pathways that resilience-building could take, and (2) making explicit and facilitating re ﬂ ection on the choices embedded in planning for urban resilience. We illustrate the tool with an application on urban ﬂ ood risk management in Rotterdam. We conclude that the Resilience Diagnostic Tool is useful to re ﬂ ect on the local goals of resilience-building, to diagnose choices made in urban plans, and to re ﬂ ect on their consequences. It supports policymakers in making deliberate, transparent and goal-oriented choices on urban re- silience.


Introduction
Resilience and 'resilience thinking' have become important concepts in both scientific research and in policy discourse (Bulkeley & Tuts, 2013;Davoudi, Brooks, & Mehmood, 2013;Eraydin & Taşan-Kok, 2013;Hegger et al., 2016;Leichenko, 2011). Resilience is interpreted as an approach, or family of approaches, that is able to cope with the high levels of uncertainty present in complex urban challenges (Wardekker et al., 2010). Resilience gained interest particularly in urban studies, such as urban planning and emergency management, partly due to its potential applicability to a wide range of urban risks and problems (O'Hare & White, 2013;Stumpp, 2013;Meerow, Newell, & Stults, 2016). Its positive connotations may also have contributed: 'strengthening resilience' provides a distinctly more positive policy framing than 'reducing vulnerability' (McEvoy, Fünfgeld, & Bosomworth, 2013). In practice, the concept has been taken up by cities and network organisations of cities worldwide. For instance, the ICLEI Local Governments for Sustainability network has been promoting resilience and organising 'Resilient Cities' congresses since 2010 (Otto-Zimmermann, 2011). '100 Resilient Cities' has been "helping cities around the world become more resilient to the physical, social, and economic challenges that are a growing part of the 21st century" (Rockefeller Foundation, 2019), for instance by stimulating the appointment of Chief Resilience Officers in cities and by providing tools and support. Similarly, resilience gained traction in recent intergovernmental frameworks, including the EU's Urban Agenda, and the UN's UNFCCC COP21 Paris Agreement, the Sendai Framework for Disaster Risk Reduction, the Sustainable Development Goals, and the Habitat III New Urban Agenda.
The resilience concept has much of its origins in ecology and complex adaptive systems research (Folke, 2006;Holling, 1973), where it is used in relation to the stability of ecosystems and the capacity of a system to recover following some shock or disturbance. It has since been applied in a wide range of scientific fields (Brand & Jax, 2007;(Leichenko, 2011), "the capacity of individuals, communities, institutions, businesses, and systems within a city to survive, adapt, and grow no matter what kinds of chronic stresses and acute shocks they experience" (Rockefeller Foundation, 2019), or "the ability of an urban system -and all its constituent socio-ecological and socio-technical networks across temporal and spatial scales -to maintain or rapidly return to desired functions in the face of a disturbance, to adapt to change, and to quickly transform systems that limit current or future adaptive capacity" (Meerow et al., 2016). For conceptual reviews, see Brand and Jax (2007), Davoudi et al. (2013), Davidson et al. (2016), and Meerow et al. (2016).
Turning resilience from a scientific concept into practical urban interventions is challenging. Research on resilience is fragmented across several disciplinary fields and its interpretation varies over time, between and within fields (Leichenko, 2011;Quinlan, Berbés-Bláquez, Haider, & Peterson, 2016). It is a fairly intangible and malleable concept, leaving much room for interpretation (Brand & Jax, 2007). Several conceptual debates have been resolved, while others remain (Cote & Nightingale, 2012;Davoudi et al., 2013;Matyas & Pelling, 2015;Meerow et al., 2016). For practitioners, the flexibility of the resilience concept may help its function as a 'boundary object', connecting the many fields, sectors and stakeholders involved in the urban system (Brand & Jax, 2007). However, it can also hinder practice, through lack of clarity and footholds to make resilience manageable and implement it in local plans. Indeed, critical literature argues that, in practice, resilience is often used as catchall term for future-proofing, without clearcut interpretation of what it means or how specific interventions or system characteristics might improve it (Davoudi & Porter, 2012;Albers & Deppisch, 2013;O'Hare & White, 2013;Stumpp, 2013). This can lead to ad-hoc policymaking, where choices taken in translating resilience to the local context remain non-deliberate, implicit, and possibly unfitting to local goals and needs.
The aim of this paper is to develop a practical tool that helps diagnose choices made in resilience-building, making them transparent and explicit, and facilitates reflection on their consequences and consistency between goals and means. In practice, policymakers and stakeholders often employ specific interpretations of what resilience means and how to improve it, despite using very broad definitions in public plans, websites and leaflets. This may often be subconscious, simply their understanding of what the broad description logically entails, viewed from their understanding of the city or from their disciplinary background. Such interpretations can also impact the decision-making tools they might use to develop resilience plans, which can have their own embedded perspectives (e.g. de Boer et al., 2009;Wardekker, in press). This can result in end-users employing decisionmaking tools without critical reflection, as black boxes that provide simple answers and solutions. Rather, we'd argue that end-users might benefit from using tools as 'learning tools' (McEvoy, van de Ven, Blind, & Slinger, 2018); as a basis for further discussion, reflection and dialogue in the city. The Resilience Diagnostic Tool presented in this paper aims to function specifically as a learning tool. It employs a broad perspective on urban resilience, combining perspectives from multiple literatures and fields and translating these to assist the user in understanding and reflecting on their perspective on resilience.
2. Approach, design rationale, and application of the tool As noted above, the tool aims to function as a learning tool, starting from a broad perspective on urban resilience, and facilitating users to reflect on their own perspectives. The tool was designed in several steps. Firstly, we examined how different branches of the resilience literature (Leichenko, 2011;Quinlan et al., 2016) conceptualised urban resilience. Secondly, we examined how these branches operationalised resilience through principles, criteria, traits, attributes, actions, and underpinning behaviours. Thirdly, we developed a framework and multi-layered set of different perspectives and principles of resilience that covered the wide range of perspectives. We explored the application of these principles in preliminary case studies in Rotterdam, London, and New York (Brown, 2016;Wardekker, 2018;Wilk, 2016). Fourthly, we developed a set of guiding steps and questions that facilitates users in using this framework to explore their perspectives and implicit and explicit choices regarding urban resilience. The final tool was field-tested in a case study in Rotterdam, as described in Section 5.
A varied spectrum of tools has been developed over the past decade to assess or measure resilience. For a review, see Quinlan et al. (2016). Many of these focus on socio-ecological systems, and occasional examples focus on urban resilience (e.g. da Silva & Morera, 2014;Tyler & Moench, 2012). Resilience measurement tools focus on pinpointing resilience in a quantitative way, for instance using models or sets of indicators (e.g. Cutter et al., 2013;Cutter, Burton, & Emrich, 2010). Potential advantages might include a high level of precision, repeatability, and easy combination with other policy tools, such as costbenefit analysis. Disadvantages might include the risk of oversimplifying resilience to a narrow set of indicators, and to only those aspects that can be readily quantified. Assessment tools, on the other hand, focus on increasing the understanding of the system and its dynamics. This stimulates learning and allows for developing a broad view on the aspects relevant to resilience, such as contextual factors and factors that are difficult to quantify. However, the results may be less clear-cut and therefore more difficult for decision-makers to process. The Resilience Diagnostic Tool is an example of an assessment tool. It combines aspects of socio-ecological and urban resilience assessment tools (e.g. da Silva & Morera, 2014; Resilience Alliance, 2010;Tyler & Moench, 2012;Wardekker et al., 2016) with those of governance assessment tools (e.g. Gupta et al., 2010;Mees et al., 2014;Runhaar et al., 2016;Verschuren & Doorewaard, 2010). This results in a tool that is particularly focused on fostering reflexivity. We expect that the tool is able to not only identify local policies and actions to increase resilience, but particularly also to elicit strategic choices that have been made, intentionally or unintentionally, in the application or planning of resilience, and to stimulate discussion, dialogue, and learning on resilience and how resilience plans relate to general urban goals. Downside is that it may not provide clear-cut answers on whether those results are good or bad and what is most necessary to change in the city's resilience planning. Consequently, policymakers may require some time to get used to this tool and its results and uses.
The tool uses a broad perspective of resilience, which is then further elaborated into general principles and more detailed operationalisations based on multiple resilience literatures. The principles were also successfully used in several cities worldwide, and other principle-based urban resilience tools (da Silva & Morera, 2014;Tyler & Moench, 2012) were successfully applied globally as well. The generic nature of such principles and guidelines, including the present one, means that they can be applied in any city, and likely also adapted to non-urban situations. The Resilience Diagnostic Tool is intended specifically to support the process that policymakers and operators go through while assessing and reflecting on the resilience of their particular case. The tool is process-based: it uses guiding steps, frameworks, and questions to perform the assessment. Therefore, it doesn't require any software or detailed data sets. As described in Step 1 (Section 4.1), the minimum input data for the tool consists of information on any policies and urban plans that relate to or impact urban resilience, as well as general information on the geography, demography, and economy of the city under study, emerging threats, vulnerabilities, local strengths and goals, and a basic understanding of the urban systems and communities. This material can be quantitative or qualitative, as it is weighed and evaluated using an expert panel. Consequently, the tool is designed to be flexibly used for a wide range of urban contexts and availability of local data.

Theory
Key observations from the literature discussed above, regarding the translation of resilience into urban policymaking, include that urban resilience is a diverse, malleable concept, involving many aspects and interpretations, and that implicit choices are often made in its local application. Choices are always necessary in translating concepts to applications; each city has its own threats, strengths, goals, and priorities. However, they have consequences and are political, and therefore should be made deliberate, transparent and appropriate.
Firstly, choices are made regarding the goals of resilience-building. This includes exploring local vulnerabilities, strengths and threats, to determine what should be made resilient against what (cf. Carpenter et al., 2001;Resilience Alliance, 2010;Wardekker et al., 2010;Wardekker et al., 2016). This is common in resilience analysis, but it also involves political choices. Considering that budget, time, and manpower are always constrained, priorities will be set on what disturbances, systems, or neighbourhoods receive attention in policymaking. More fundamentally, critical resilience literature has started exploring how actors interpret and frame urban resilience. For instance, Davoudi and Porter (2012) observe in policy documents and discussions in the UK, that resilience is interpreted primarily as 'self-reliance' and people's innate ability to handle problems, while seeing strong governmental risk management as a threat to this resilience. Others note differences in framing of urban resilience as 'quick recovery' to equilibrium versus 'adaptive/evolutionary' resilience focused on moving with changes (Davoudi et al., 2013;Sakai & Dessai, 2015;Wardekker, in press), whether the focus is on urban systems versus communities (Wardekker, in press), or on resilience 'against anything' or disturbance-specific resilience (Matyas & Pelling, 2015). These goal-related choices exhibit different (implicit) preferences and emphases for resilience-building: they determine who, what, when, where, and why resilience-building takes place (cf. Meerow et al., 2016;Meerow & Newell, 2019). This determines what aspects of resilience are relevant.
Secondly, choices are made regarding which aspects of resilience are improved. A sizable section of the theoretical and analytical resilience literature has attempted to flesh out what resilience means, going beyond definitions. They inventoried traits, attributes, actions, and behaviours that underpin resilience, often in the form of 'resilience principles'. They describe "specific mechanisms and behaviours that make a city resilient or that help policies and practices improve resilience. They can focus on the system, or its governance." (Wardekker, 2018), and are often generic and thus applicable to different situations. Such principles have been applied to design and evaluate policy options and plans for resilience, for a variety of issues and systems (e.g. Barnett, 2001;Biggs et al., 2012;Tyler & Moench, 2012;Wardekker et al., 2010;Wardekker et al., 2016;Sharifi & Yamagata, 2016;de Bruijn, Buurman, Mens, Dahm, & Klijn, 2017;Heeks & Ospina, 2019). Examples include: buffering, redundancy, and flexibility. See Supplement S1 for comparisons. Individual sets of principles range from generic to very specific aspects, and often have their own specific focus (e.g. on systems, governance, community; reactive or proactive; short-or long-term). We synthesised these, based on support by multiple sources, into a layered framework. See Table 1 for a detailed overview of the analytical procedure that we followed to conduct this synthesis. See Fig. 1 and Table 2 for our final framework (Supplement S2 contains further detailed descriptions), from generic pathways to general principles, to specific operational criteria. Similar to Sharifi and Yamagata (2016), we categorised them based on Linkov et al.'s (2014) notion that resilience can relate to planning for, absorbing, recovering from, and adapting to disturbances. These categories are useful, because they highlight different pathways for resilience-building (e.g. proactive, reactive, recuperative, adaptive). They leave room for different priorities and approaches. They also address human agency, which tends to be neglected in current scientific and practical resilience approaches (Davoudi et al., 2013). Analysing how cities address the resilience principles would show what aspects of resilience are actually improved or diminished.
Thirdly, choices will have consequences. Some things are improved, while others are not. Cities may want to focus on specific resilience principles, rather than the full set. Their priority depends on the local situation. Implicit choices might show inconsistency between the emphasized principles and the goals; choices may not be goal-oriented. Deliberate choices would likely improve this alignment. Furthermore, choices can have side-effects. They could involve trade-offs, for instance in balancing resilience between time scales (short/long term), spatial scales (neighbourhoods, city, region), system components (specific infrastructures, sectors) (Chelleri et al., 2015;Olazabal et al., 2018). Conversely, resilience-building may involve synergies and co-benefits with other urban goals. These aspects determine who benefits from resilience-building and who does not, involving matters of politics and justice (cf. Bahadur & Tanner, 2014;Cote & Nightingale, 2012;Meerow & Mitchell, 2017). This is important information for the broader political and societal appropriateness of resilience-building. Consequently, the (un)intended consequences of choices should be made transparent, and their appropriateness should be deliberated.

Resilience diagnostic tool
The Resilience Diagnostic Tool employs a three-step approach, building on Section 3. The first step reflects on choices in the goals of resilience-building by examining the local situation and goalsetting. The second explores choices made: which aspects of resilience (resilience principles) are emphasized? This is done for both the current situation (baseline) and proposed plans, measures, or policies for resilience-building (interventions). The third reflects on consequences of these choices: whether the interventions match the goals and potential side-effects. Lastly, follow-up is considered. We developed the elements within these steps based on planning support tools (Gupta et al., 2010;Mees et al., 2014;Resilience Alliance, 2010;Runhaar et al., 2016;Verschuren & Doorewaard, 2010;Wardekker et al., 2016). Often-distinguished elements include: preparation, problem diagnosis, resilience assessment, and intervention design or evaluation. See Fig. 2. Step 1 collects preparatory steps, such as delineating the scope of the study, determining the goals, 'setting the scene', inventorying context, first exploration of the relevant disturbances (based on Resilience Alliance, 2010;Meerow et al., 2016;Mees et al., 2014;Runhaar et al., 2016;Wardekker et al., 2016). Taking this step explicitly is important.
Step 3 is a reflexive step that is introduced with this tool. It is less developed in current assessment tools. It builds on calls in the literature on reflect on the resilience goalsetting (the 'why' of resilience) (Meerow et al., 2016;Meerow & Newell, 2019;Meerow & Mitchell, 2017), resilience trade-offs (Chelleri et al., 2015;Olazabal et al., 2018), and resilience framing (McEvoy et al., 2013;Sakai & Dessai, 2015;Wardekker, in press). Finally, Step 4 briefly reflects on follow-up work, which can range from communication of the results to added research or addressing any discrepancies or problems found (based on Resilience Alliance, 2010;Wardekker et al., 2016;Runhaar et al., 2016). Accordingly, the tool diagnoses and makes explicit the choices on goals, choices on emphasized aspects of resilience, and their consequences.

Step 1: preparation and goalsetting
Before any resilience diagnosis is conducted, it is important to delineate what's important in the local situation and context, and consequently what the goals for resilience-building might be. A. Wardekker, et al. Cities 101 (2020) 102691 Table 1 Analytical stages that we followed in designing the final framework of resilience principles (as presented in Fig. 1 and Table 2). For details see Supplement S1 and S2 and Wilk (2016)

Unit of analysis, information sources, and participants
The unit of analysis should be determined (cf. Runhaar et al., 2016). The diagnosis could focus on for example a specific project, program, or policy document, or a wider variety of urban plans, strategies, and policies related to a specific topic. These could be public or private plans, or a combination.
The tool is process-based, requiring discussion and information in different steps. This can vary depending on the background and expertise of participants and the focus of the analysis. Input information can be gathered from existing information (studies, policy documents), new research, and expert or stakeholder input. Potential methods include: document analysis, measurements and monitoring, modelling or indicator studies, interviews, expert panels, workshops, and focus groups.
The tool is designed for use by a team of analysists, such as municipal specialists, advisory bodies, consultancies or scientists. Scoring will require participants with expertise in resilience, planning, and specific locally-relevant topics (water, housing, social issues, etc.). Results can then be used by a broader spectrum of local policymakers to critically reflect on their plans and approach, as they indicate potential gaps in the way their plans cover resilience and mismatches with intended goals. It is important to reflect on the desired and practical level and form of participation of local actors, such as policymakers, companies, NGOs and citizens. Fully participatory analysis is possible, but time-intensive. Conversely, diagnosis by an outside expert group using existing information is faster, but risks missing important local processes, interests and details.

Inventory of case context
An overview of the local situation is required. Detailed analysis can be time-consuming, but often this information is already available. Quick analysis could involve a short meeting or workshop with participants from different departments to inventory materials, and conduct a SWOT-analysis. Relevant aspects include: (a) The area, system(s) and communities under study: Gather information on the past, present, and future of area and it's systems and communities. Resilience is a flexible concept and it is important to define explicitly what the specific goals and focus is for resilience-building, given the context discussed above: resilience of who, what, when, where, why (Meerow et al., 2016)? Different actors will have different interpretations, goals, and priorities: why are we building resilience? It also includes demarcations: spatial scale (street, neighbourhood, city district, city limits, metropolitan area, etc.), time (coming years, decades, or longer), and topic (specific disturbances, sectors, or 'resilience of everything against everything').

4.2.
Step 2: diagnosis of emphasized aspects of resilience Step 2 diagnoses which choices are made in resilience-building. Particularly: which aspects of resilience are emphasized, and which not. These aspects are defined in Table 2 (detailed descriptions in Supplement S2). We recommend performing the diagnosis at the level of resilience principles, using the following procedure. First, the principles are translated to a localized interpretation. Secondly, the current situation is compared to this, to diagnose current strengths. Thirdly, the proposed plans, projects, or other interventions are compared, to diagnose the choices made in them.

Translating resilience principles to localized resilience narratives
Resilience principles are fairly generic and conceptual. Even when clearly defined, they require some interpretation to meaningfully relate them to the local context. It is useful to translate them into short narratives that describe what an application of that principle would look like in the city, particularly in relation to the goals and disturbances defined in Step 1. The operationalisations in Table 2 (column 3) can be used for this. These narratives provide something to compare and score against: How far is the city from that narrative (baseline diagnosis), and how do the plans impact those narratives (intervention diagnosis)? See Supplement S3 for examples.

Baseline diagnosis: current emphases regarding resilience
The current situation is scored, keeping in mind the demarcations set in Step 1 (e.g. time horizons, spatial scale, topic). If no adaptation takes place, how would the city, systems, and communities respond to current and future climate-related disturbances? Examine both shocks (short-term events) and stresses (long-term pressures). To what extent does the city have the attributes described by the framework? Score each resilience principle, using the scales in Table 3. We caution against uncritically averaging or aggregating scores, because relative importance of principles varies with context, and the goal is exploring emphases rather than assigning a final resilience-'grade' (see also Wilk & Jonsson, 2013). Note that the second sample scale (weak/strong and negative/positive effect) seems easier to understand than the first (little/much emphasis), but can also come across to participants or people reading the results as a 'grading' scale, describing whether the city is doing well or not. If used, one should carefully explain that this is not the intent. The scoring procedure would follow a structure, per resilience principle, such as: 1. Compare the material inventoried under step 1 (policy documents, research materials, interviews, etc.) with the resilience narrative developed for this principle. 2. Discuss the strengths, weaknesses, and opportunities of the current situation with regard to this principle. 3. Collect arguments for both a high score and a low score. In many cases, the situation will be nuanced, and there will be evidence for both strong and weak points regarding a principle. It is important to explicitly weigh these. 4. Assign a score, using one of the scoring scales below (Table 3). This can be done either as a joint decision by the group of analysts doing the analysis (consensus score). It can also be done individually: scoring separately, followed by comparison, further argumentation,  (Boyd et al., 2015;Cutter et al., 2010Cutter et al., , 2013Davoudi et al., 2013;Eraydin & Taşan-Kok, 2013;Godschalk, 2003;Gunderson, 2009;Norris et al., 2008;Schelfaut et al., 2011;Tanner, Mitchell, Polack, & Guenther, 2009;Tyler & Moench, 2012;Wardekker et al., 2010 (Barnett, 2001;Biggs et al., 2012Biggs et al., , 2015Nelson, Adger, & Brown, 2007;Wardekker et al., 2010;Wardekker et al., 2016) Absorb: Absorbing disturbances (ability to dynamically cope with disturbances that take place, maintaining desired functions)

Intervention diagnosis: impact of plans on resilience principles
Diagnosis of the impacts of plans (or policies, projects, etc. as defined in 'unit of analysis' in Step 1) on urban resilience is performed in a similar manner, with the scoring scale in Table 3. Plans may improve one principle, while reducing others. They may also result in meaningful adaptation, while not improving resilience (e.g. improving resistance instead). If plans for different sectors or topics have very dissimilar results, consider doing separate diagnoses. The scoring procedure for interventions would be similar to that for the baseline, except that it should reflect on how the plans might change the situation described for the diagnosis. Note that the analysis could focus on eliciting the 'direction of change' (as suggested in Table 3) or on the 'end result'. The latter is likely very difficult to establish, unless detailed designs or assessments of the impacts of policy interventions are available. The former is easier to perform using policy plans, interviews and expert panels.

Supplementary diagnosis (optional)
Other aspects may need to be examined more closely and scored separately for comparison to the baseline and intervention diagnoses. For example: (a) changing contexts, including trends in other sectors, at lower or higher spatial or governmental scales, or in connected systems; the influence of autonomous adaptation; (c) relative priority of specific resilience principles as seen by the resilience team or by different societal groups.

Step 3: reflection on consequences of choices
This step explores the outcomes and consequences of the diagnosis. See Supplement S4 for tables that can be used to support this process.

Consistency of resilience emphases with local situation and goals
The diagnosis can now be critically examined: what principles are emphasized (high-scoring), and which not? Which pathways are covered? Resilience-building takes time, effort and funding, and priorities will need to be set. Some cities may strive to cover all principles or pathways, others may not. This depends on the specifics of the situation, including local disturbances, vulnerabilities, socio-economic situation, different urban actors' goals and aspirations, and the political situation. For example, cities faced by limited short-term shocks may emphasise recovery afterwards, while those facing major long-term trends might emphasise adaptability. Also note that some principles, such as flexibility and diversity, might be better than others at helping the city prepare for 'unknowns'; surprise events or trends (see discussion in Wardekker et al., 2010).

Potential side-effects (trade-offs, co-benefits) of interventions
Resilience-building will likely have side-effects, whether positive, negative, or neutral. Co-benefits may occur in other policy fields, economic sectors or at other scales. For example, community gardens may improve rainwater buffering, but also increase social cohesion, health and food access. Information on this can be obtained from policy documents, knowledge among participants, or stakeholder consultation. Trade-offs will be made as well: adaptations benefit some, while disadvantaging others. Chelleri et al. (2015) discuss resilience trade-offs between time scales, spatial scales, and system components. An example of a time scale trade-off is improving preparedness against shortterm shocks through capital-intensive or hard-engineering approaches, which may reduce flexibility and adaptability for long-term issues. Other trade-offs may be present, such as between different policy goals or domains, or between specified resilience (against specific impacts) and general resilience. Side-effects can be compared to the goals of resilience-building inventoried in Step 1 (contextualising resilience): are they in line? Wider debate may be needed to evaluate political and societal acceptability of side-effects.

Follow-up
The final step explores: 'what next?' Any diagnosis involves limited resources (including information), and can yield new questions. Additional studies may be required, detailing specific topics or neighbourhoods. Similarly, it could be useful to revisit the normative basis (Step 1) and implications of resilience-building (Step 1) with a broader group of societal actors. Finally, disseminating the diagnosis into societal and political decision-making may require stakeholder analysis and further tailoring, visualisation and communication.

Illustrative case study: flood resilience in Rotterdam
We conducted an illustrative case study on flood resilience in the context of climate adaptation in Rotterdam, the Netherlands. Rotterdam has been working on resilient climate adaptation for over a decade (e.g. Wardekker et al., 2010), is active in resilience innovation and exploring resilience knowledge (Ilgen, Sengers, & Wardekker, 2019), and was among the first to publish its Resilience Strategy (Municipality of Rotterdam, 2016). Consequently, much information Fig. 2. Steps of the resilience diagnostic tool. A. Wardekker, et al. Cities 101 (2020) 102691 Table 3 Sample scoring scales for diagnosing which resilience principles are emphasized.

Generic emphasis scale
There is very little to no emphasis on this principle. Baseline: The situation is very dissimilar to the aspects described.
Interventions and plans: There is very little attention, and/or very few measures directed at this aspect. There is little emphasis on this principle.
Baseline: There are some, but few similarities to the aspects described.
Interventions and plans: There is little attention, and/or few measures directed at this aspect.
There is limited emphasis on this principle. Baseline: There are similarities, but not much, or mixed overall picture. Interventions and plans: There is limited attention, and/or some measures directed at this aspect.
There is much emphasis on this principle. Baseline: The situation is similar to the aspects described.
Interventions and plans: There is much attention and/or many measures directed at this aspect There is very much emphasis on this principle. Baseline: The situation is very similar to the aspects described.
Interventions and plans: There is very much attention, and/or very many measures directed at this aspect.

Separate scales for baseline and interventions
The current situation is very weak regarding principle.
There are key weaknesses on most aspects, no strengths. Opportunities may be missed. The current situation is weak regarding principle.
Either overall weak or mix of weaknesses and strengths that is still largely unfavourable.
The current situation is neutral regarding principle.
Mix of strengths or opportunities and weaknesses, with overall neutral or unclear effect.
The current situation is strong regarding principle.
Either overall strong or mix of weaknesses and strengths that is still largely favourable.
The current situation is very strong regarding principle.
There are key strengths on most aspects, no weaknesses and possibly valuable opportunities.
The adaptation plans have a strongly negative effect on this principle.
It has a negative effect on most aspects/ The adaptation plans have a negative effect on this principle. Either an overall negative effect, or positive on some aspects/operationalisations while neutral or The adaptation plans have no or a neutral effect on this principle. A. Wardekker, et al. Cities 101 (2020) 102691 and experience was available, presenting a valuable opportunity for testing the tool. Data was collected using document analysis of policy documents spanning climate adaptation, water management and resilience, 26 interviews with key policymakers and stakeholders, participant observation during a research stay, and two workshops (full list in Supplement S5). The first workshop focused on step 1-2 and involved six social & natural science researchers, covering urban resilience, governance, spatial planning, ecology, and sustainable development. The second workshop focused on step 3-4 and involved four researchers and four practitioners from Rotterdam's Resilience Team and Delfland Water Authority. We also jointly reflected on the applicability, usability and role of the tool.

Unit of analysis and inventory of plans
We focused on flood risk management, from rivers, sea, and rain, including major (flood safety) and minor flooding (flood nuisance). The unit of analysis was at city-level, diagnosing current flood resilience and the impact of current municipal plans on these. We examined four formal municipal plans: Rotterdam Water Plan II, Rotterdam Climate Adaptation Strategy, Rotterdam Climate Proof, Rotterdam Resilience Strategy (Municipality of Rotterdam et al., 2007, Municipality of Rotterdam et al., 2013Municipality of Rotterdam, 2013a, 2013b. 1 Informal pilot plans were included when explicitly indicated in these.

System(s) under study, key disturbances, trends, vulnerabilities, current strengths
Key threats are related to flooding from intense precipitation, cloudbursts, river and sea. Heat and sea level rise are also concerns. Flood protection standards are high and structural flood defenses are in place. Consequently, flood probability from rivers and sea is currently low. However, potential impacts are high: much of the city is below sea level, capital-at-risk is high, and there are vulnerable areas (polders, unembanked areas) and many critical infrastructures. Probability of cloudburst-related flooding is higher, but impacts are less severe: no casualties, though potentially substantial economic damage. Several potential aggravating factors were identified.

Goals for resilience-building
The plans approach resilience from a positive view on living with water; vicinity to water is not only a threat but also an opportunity. The municipality seeks to establish itself as an exporter of Dutch water solutions worldwide and a leading innovation lab for designing solutions to live on and with water. Rotterdam is a 'new city', largely levelled during World War 2. This allows for fast (re)developments and space for developing and implementing novel solutions. This echoes with observations (cf. Brown, 2016;Wilk, 2016) that Rotterdam emphasises economic aspects, particularly business, the port, water, innovation and exporting innovations. For instance, Rotterdam is currently involved in exporting its 'water squares' to Mexico-City (Ilgen et al., 2019).
Resilience is frequently mentioned as 'anchored in the DNA' of Rotterdam's population. It therefore has a strong appeal and legitimation. Rotterdam's Resilience Strategy focuses on enhancing overall resilience, for everyone in the city, covering a broad range of topics: society, the port, clean energy, cyber, climate adaptation, infrastructure, networks, and 'anchoring' of resilience (Municipality of Rotterdam, 2016). However, increasing focus seems to be paid to citizens and communities, previously identified as weakness. The "21st century" is the timeframe of the Resilience Strategy. Other policy documents mention 2030 (Municipality of Rotterdam, 2007; Municipality of Rotterdam et al., 2007, Municipality of Rotterdam et al., 2013) or 2035(de Greef, 2005 specifically.

Resilience diagnoses
We diagnosed the current situation (baseline) and available plans (interventions). First, general strengths, weaknesses, opportunities and threats were discussed. Second, we discussed per principle the current situation, climate change impacts, and adaptation and water plans. Third, we compared these to the developed resilience narratives (Supplement S3), and discussed and assigned scores per principle. See Fig. 4. Participants' argumentations for scores will be briefly discussed below. For detailed argumentations, see Supplement S6.

Baseline diagnosis: current flood resilience in Rotterdam
Rotterdam is strong in anticipation & foresight through strong knowledge networks, complementing municipal research, streamlined knowledge development and documentation of (local) climate-related risks. Another emphasis is redundancy, particularly in the water system (e.g. compartmentalisation, emergency backups), municipal structure, and water and adaptation governance (e.g. combining public and private action). Rotterdam has a strong tradition of learning: A. Wardekker, et al. Cities 101 (2020) 102691 experimentation, innovation, and learning from experiences. Institutional memory is also present through long-term civil servants with expertise on water, adaptation and resilience.
Other principles receive less focus, such as preparedness & planning (of the public, businesses, and other organisations). Public awareness of flood risk is very low. Communication efforts are fragmented, but improving. Flatness is also low; broad stakeholder engagement and inclusion in adaptation implementation is limited. Citizen involvement is strongly project-dependent, but improving. There are several barriers for flexibility, for instance in budgetary flexibility, path-dependency towards hard, structural interventions, and limited spatial flexibility due to high competing spatial claims. Rotterdam has high cultural diversity, but low spatial diversity of critical functions and limited differentiation of functions.

Intervention diagnosis: how do adaptation plans impact flood resilience?
Decisive changes are expected for preparedness and flatness, especially due to strong emphasis in the Rotterdam Resilience Strategy on improving public flood risk awareness and education, and community involvement in water-related projects. Interviewees indicated that citizens increasingly take responsibility for adaptation and self-determined actions (Wilk, 2016). Community activities could also strengthen social cohesion and networks, whose resources can be rapidly accessed (high flux). This planned focus has recently also been implemented: in the program Water Sensitive Rotterdam (WSR, 2019), the municipality paid strong attention to citizen participation and community activities. Other improvements are expected in cyber resilience, and business preparedness for adverse events. Improvement of redundancy is expected in the water system, ICT infrastructure and underground critical infrastructure. Small-scale green infrastructure, renaturation projects, and 'build with nature'-approaches improve ecological, heat and flood resilience by stabilizing feedbacks mechanisms (homeostasis).
Contrastingly, based on the plans, few changes can be expected for diversity and flexibility. The issues presented in the baseline diagnosis are present here as well. Participants argued that they were difficult to influence by the municipality.

Reflection
A structured reflection on the resilience diagnoses (Step 3) and the tool itself, guided by a set of questions, occurred in the second workshop, involving local policymakers.

Emphases and non-emphases in the adaptation plans
The current situation is highly targeted on anticipation, redundancy, and learning. These cover three pathways: planning & preparation, absorbing impacts, and adaptability, albeit with one resilience principle each. The adaptation and resilience plans entail a much broader approach, towards all-round resilience. They cover all pathways with multiple resilience principles. Improving flatness (pathway: Recovery) is a particular emphasis; it is strongly improved. Two principles not covered, diversity and flexibility, were difficult to influence by the municipality. The identified emphases were in line with the expectations of Rotterdam's resilience team and water authority. The Resilience Strategy focuses on a broad resilience approach. Citizen inclusiveness was indeed a former weakness that is already a point of attention. Consequently, the tool successfully allowed an outside team of experts to assess the situation in Rotterdam. Although the presented results were unsurprising to local experts, Step 2 can usefully organise available knowledge, particular in cities that are less advanced in developing resilience strategies.
Some discussion ensued on specific scores for rainwater storage capacity (robustness & buffering). Local experts argued for a higher score, observing that rain resilience is a strength compared to heat stress. Measures were taken and Rotterdam no longer focuses purely on 'cubic metres of rainwater to be stored'. Instead, a level of residual nuisance is accepted. This is an explicit change in risk evaluation; a trade-off is made between costs and acceptable impacts. Participants also misinterpreted the original scoring scales and visuals; it seemed as if the plans decreased anticipation (in total), whereas they only emphasise it less strongly compared to now. We adapted the scales and visuals to better highlight that they involve emphases. We observe that it is important to discuss Step 2 with local experts when assessing with an outside team, and that local experts may still interpret scores as 'grades' and turn defensive regarding low scoring principles. Another discussion focused on the level of application. Despite Rotterdam's broad resilience approach, the focus on full-city water resilience was deemed too broad for local use. Participants preferred to assess specific topics, such as cloudbursts or water safety, separately. Practitioners often work on small parts of the broad field of water. For politicians, the tool should ideally indicate whether and where improvements are required, instead of being a benchmark. Visualizations comparing different neighbourhoods and/or topics would help. They can communicate and advocate specific challenges (e.g. embanked vs. un-embanked areas) and strengths/weaknesses, and it is easy to explain that different topics and neighbourhoods require different emphases.

Trade-offs, co-benefits and (un)intended side-effects of adaptation plans
We discussed side-effects using Chelleri et al.'s (2015) trade-offs: time scale, spatial scale, and system component, adding an 'other trade- Fig. 4. Diagnosis of the emphases in the baseline (left) and adaptation plans (right) for Rotterdam's flood resilience. The figure shows a highly targeted current situation, whereas the plans take a broad approach. A particular large change can be observed for flatness. Scale as in Table 3. A. Wardekker, et al. Cities 101 (2020) 102691 offs' category (e.g. between concepts/goals, such as sustainability, climate mitigation; Johnson et al., 2018). Participants had difficulty grasping the notion of trade-offs and relating it to concrete resilience measures. We shifted to discussing 'paradigm shifts', which was easier to grasp. Such shifts inherently have trade-offs, co-benefits, and other consequences. Participants observed two key paradigm shifts. Firstly, there was a shift in perspective from treating water as threat to treating water as opportunity. This implies a trade-off by transitioning from preventing impacts from short-term shocks, to accepting some impacts while adopting a long-term view on living with water and sea level rise (absorb, recover, adapt). By being among the first to experiment with long-term adaptation solutions like floating buildings, Rotterdam provides future-proof solutions for living on water for its population and also strategically creates economic co-benefits by exporting these worldwide. In addition, this mind shift has opened up new ways of collaboration and co-creation across policy domains in an area previously dominated by technocrats.
The second paradigm shift was anchoring broad resilience-thinking by promoting the adoption of a resilience lens on all kinds of issues. This implies a trade-off moving from a narrow application on specific topics, which makes it easier to measure, discuss and agree among specialists, to broader societal debates where different priorities and resilience approaches compete. However, stimulating resiliencethinking can result in positive knock-on effects in several policy domains and system components. For instance, investments in broadening education options to foster citizen entrepreneurship could improve wealth equity in the long run. Creating self-sufficient city districts, including urban farming initiatives, could render citizens more independent consumers and improve health and social cohesion as well. If these developments were to benefit several parties (e.g. businesses, citizens), the overall commitment to resilience-building could be strengthened.

Follow-up: diagnostic information in local practice
The preferred tools to discuss and evaluate the resiliency of projects, processes or city districts are a continuous development, in which scientific research and daily practice collide with local politics. The Rotterdam municipality further developed a Resilience Scan. Similar to the Diagnostic Tool, it uses resilience principles, but it is more assessment focused, using lists of analytical questions, specifically at the level of projects. In comparison, the Diagnostic Tool focuses on reflection and reflexivity on key choices, at a strategic level. Both tools can learn from each other, but have different focus and applications. Rotterdam's definition of resilience is the ability of people, communities, organisations, businesses and systems to survive, adapt and grow, regardless of long-term tensions and crises of all kinds and varying magnitudes. Resilience becomes a key urban quality to benefit from and to cope with transitions that can cause stresses or shocks. Sustainability, social, cyber and urban resilience should be taken into account. Among others, city planning of the districts 'Feyenoord City' and Merwe-Vierhaven used the resilience scan and other tools. The result is a hands-on approach in which actors are encouraged to become more resilient. In local politics, clean energy, cyber, climate adaptation and 'anchoring' of resilience are the assumed priorities for the next four years. Various departments and initiatives within the municipality will work together with institutes and public-private initiatives on solving these wicked problems. Residents also play an important role. These actors will strengthen each other in creating a more resilient living environment. Daily practice, however, requires factual substantiation for legitimacy. Diagnostic information, such as provided by the Resilience Diagnostic Tool, stimulates fruitful symbiosis between multidisciplinary research and local implementation of resilience as a concept. This improves the substantiation of local resilience-building.

Discussion & conclusions
We presented a diagnostic tool for reflecting on urban resilience. Urban resilience is a complex notion, involving many aspects and choices. The tool differs from other methods by using a multi-layered set of resilience principles, synthesised from multiple literatures, and by explicitly focusing on choices. We illustrated it through a case study on flood resilience in Rotterdam.
The conceptual and analytical literatures on urban resilience have made much progress over the past decade in fleshing out what resilience means, including definitions, system principles, governance principles, criteria, and indicators. Recently, however, emerging critical literature has observed that various scientific fields emphasise different aspects of resilience, and that in urban practice, definitions and choices are often left implicit. This could lead to ad-hoc non-deliberate design of resilience interventions. Choices can be highly political in nature, involving normative aspects such as power and equity. We argue that choices are an unavoidable and necessary aspect of translating a scientific concept to local interventions. Cities face different challenges and have different strengths, tools, and resources at its disposal. While one city may want to focus on strategic anticipation and long-term flexibility, another may have pressing short-term needs and rather focus on absorbing shocks and recovery afterwards. The Resilience Diagnostic Tool therefore focuses not on resilience assessment or standardized resilience scores, but on making the choices regarding resilience principles transparent and explicit. It consequently enables informed choices on which principles matter most for the local experts and their stakeholders. The tool does not examine normative aspects, power, and equity themselvesthis would be valuable, but would require a dedicated approach. Rather, we build on the existing rigour of the conceptual literature by synthesising and operationalising resilience principles from multiple fields, covering multiple directions that resiliencebuilding could take. We expand on this by reinterpreting resilience definitions and principles as a matter of a local choice spectrum. We embedded this within a series of steps, drawing on the critical literature, to diagnose choices made and directions taken or not taken, and stimulate reflection on their appropriateness and consequences. The tool addresses resilience from a broad perspective, covering interpretations from multiple literatures. This includes classic resilience principles, such as buffering and redundancy, but also the importance of social and institutional ones, highlighting that the capacity to cooperate and coordinate resilience actions between city departments, different stakeholders, and citizens in making choices on resilience is a key aspect of urban resilience. As such, the paper contributes to the scientific literature on urban resilience by providing a new look at choices made in resilience-building and a tool to facilitate this.
Practically, the tool helps participants collect, organise, aggregate and assess available material on local disturbances and interventions, by separating the diagnosis into a clear step-by-step process, from goals, to current situation, plans for interventions, and potential consequences. The case study showed that a group of outside experts could meaningfully explore details and complexities of resilience for a city. Results were validated by local practitioners. The tool also stimulates resilience-thinking along multiple pathways, and how different interventions might or might not improve these. This helps establish a more explicit, broader reflection of what resilience means, in local context, and what choices might be made. For well-established specialised municipal resilience teams, the scoring may provide unsurprising results, but the process itself and the explicit reflection are valuable. For less experienced teams or in organisations with fragmented expertise, Step 2 would be particularly valuable. The practical value of the tool therefore is that it stimulates policymakers and other actors to explicitly and transparently think in terms of choices and their consequences and appropriateness.
In the illustrative case study, we learned several lessons on how to the tool can be applied and refined. Firstly, it can be applied at a A. Wardekker, et al. Cities 101 (2020) 102691 general, holistic level (all-round resilience, city scale), or specified per topic and/or neighbourhood. Participants observed that for holistic applications, resilience principles aggregate many different aspects over many topics. Consequently, much background information is needed to properly score and interpret the results. More specific analyses are easier to perform and communicate, though they could neglect interactions between subtopics. Secondly, interpretation, operationalisation, scoring and weighing resilience principles are unavoidably subjective. We partly accounted for this through team-based scoring and using storylines as reference point. Further societal discussion and co-design (cf. Sharifi et al., 2017) could be beneficial. Thirdly, more work is needed on making trade-offs and co-benefits easier to grasp and identify. A starting point could be to breakdown plans into major goals or themes, and disaggregate these into actions, for which it might be easier to identify concrete trade-offs and co-benefits. Fourthly, further work is needed on how to best communicate and visualise results of resilience diagnoses to politicians and public. Participants indicated that these groups preferred visuals that directly highlight what needs to be done, which is not readily apparent. Starting point could be to visualise other steps in the tool using tables and causal models. Ideally, however, visually approaches would be developed to contrast the (actual) emphasized aspects of resilience with the (intended) goals of resiliencebuilding. Regarding the limitations and downsides of this tool, as expected, participants needed some time to get used to the tool and the reflexive nature. One participant noted that it would be much easier if the tool would provide a clear-cut answer on where or what should be improved since that type of information is much easier to use in conversations with politicians. Other tools, particularly resilience analysis tools, are generally more suited to providing such information. Regarding the benefits, the case study showed that the tool did indeed allow users to trace the strategic choices that had been made in the past and in current policy plans. The results of the external team that did the analysis in the first workshop, based on policy documents and interviews, was confirmed by the local policymakers in the second workshop. The tool also successfully kickstarted a discussion on the nature of these choices, why they were made, and what implications they might have, even though discussion of trade-offs was challenging. Further research would be needed to evaluate how effective it is in stimulating discussion between different actor groups. All things considered, the Rotterdam case study showed that the tool successfully stimulated a reflexive assessment and allowed for explicit discussion on choices made in policymaking on urban resilience. To summarize, the Resilience Diagnostic Tool structures the diverse facets of urban resilience and provides a practical and locally situated diagnosis of the choices that cities make in promoting these. In particular, it stimulates thinking on resilience-building in terms of choice space and emphases, and their appropriateness to the local situation. This helps scientists and practitioners have a more comprehensive, and reflexive debate, and supports policymakers in making deliberate, transparent and appropriate choices on urban resilience.

Declaration of competing interest
None.