Climate and disaster resilience measurement: Persistent gaps in multiple hazards, methods, and practicability

In response to increasing demands for information on disasters and extreme events by the policy, practice, and research communities, there has been a recent surge in approaches to the measurement of applied risk management and resilience. Nevertheless, very few of these approaches address systemic risks, particularly in multi-hazard environments, and thus do not holistically contribute to decision making in various contexts. This paper addresses this gap by means of a critical review and an assessment of approaches to climate and disaster resilience measurement with a particular focus on three issues: (1) the consideration of compounding socioeconomic and climatic risks in approaches to resilience measurement; (2) the methodological and technical aspects of resilience measurement; and (3) the application and practicability of resilience mea- surement across various contexts to reliably inform decision-making processes. Seventeen key resilience measurement approaches developed by researchers, government, and private and civil society organizations are selected and evaluated according to a set of assessment criteria. Based on this assessment, we conclude with three key findings. First, we find a lack of clear standards and validated approaches in the measurement methodologies, which can lead to inconsistencies and poor data comparability. Second, approaches to resilience measurement should further strive to combine both process- and outcome-based methodological perspectives to represent resilience in the most holistic and standardized manner possible. Third, in the context of multiple hazards, decision-making strategies should address multiple vulnerabilities. To conclude, we suggest that future developments in resilience measurement should allow for the analysis of interactions be- tween multiple stressors across different scales and among systemic risks. Moreover, more rigorous process-based approaches to resilience measurement are still required that can incor- porate outputs into decision making.


Introduction
In response to the need for information by the research-practice-policy communities on disasters and extreme events, there has been a surge in the development of risk management and climate adaptation methods, frameworks, and implementations (Berrang-Ford et al., 2021;IPCC, 2022;Salehi et al., 2019). Climate and disaster resilience measurement approaches are no exception and have progressively garnered interest because of the uncertainties and complexity caused by interconnected climatic and social risks drivers (Aldunce et al., 2015;de Bruijn et al., 2017;Nalau and Verrall, 2021;Schinko et al., 2017). Moreover, the COVID-19 crisis has shed new light on resilience as a systems-based concept, defining it as a policy goal for mobilizing investments in compound events and future crises in an increasingly interconnected world. Now, more than ever, a holistic conceptualization of resilience that can inform concrete short-and long-term responses at scale is needed (Phillips et al., 2020;Schipper et al., 2021;Trump and Linkov, 2020).
Within the spectrum of climate risk management methods, measurement of climate and disaster resilience is a risk-based approach that goes beyond managing direct risks. Instead, it takes into account multiple dimensions, such as enabling conditions, capacities, and critical functions (services), thresholds, and cross-scale (both space and time) interactions to cope, adapt, or transform before and after a disaster or climatic shock (Connelly et al., 2017;Winderl, 2014).
As a climate risk management approach, resilience measurements have been introduced as a promising way of tackling flood disaster risk challenges under non-linearity and uncertainty (de Bruijn et al., 2017;Hochrainer-Stigler et al., 2021). In other words, resilience measurement approaches often manage complexity via eliciting and generating different levels of knowledge (Lavell et al., 2012) that consider different framing and decision-making contexts (Berkhout et al., 2013;Schinko et al., 2017)as well as the information needs of stakeholders in the field (de Bruijn et al., 2017;Keating and Hanger-Kopp, 2020). Moreover, the context-specific characteristics of such approaches (Laurien et al., 2020) incorporate multiple relevant aspects needed to support the integration of the research-policy-practice communities across different scales (Keating and Hanger-Kopp, 2020). Many resilience tools and other services to inform decision-makers have been developed and used by local and national governments, NGOs, academia, the private sector, and multi-organization collaborations to measure the resilience of countries, communities, and cities against climate risks and to support decision making under uncertainties and complexity (Cai et al. 2018). The relevance of this topic, for example, has also grown on the international level both in the development context (see UNDRR's (2017a) resilient cities initiative to support the implementation of the Sendai framework) and the climate change strategy (eg. EU's adaptation strategy started calling for resilience services and decision-support approach on adaptations to climate change (EC, 2021)). When it comes to applications, practitioners, such as spatial planers, urban consultants, and disaster risk officers, will need to make a decision about how to gain climate and disaster risk information; the challenge and barriers, however, are usually to decide which approach is most applicable for practice and policymaking.
Despite the wealth of existing approaches to resilience measurement services and the increasing body of literature reviewing them, most reviews have only focused on the definition and concept of resilience (Béné et al., 2017;Cai, 2020;Ceré et al., 2017;Keating et al., 2017a;Mayunga, 2007;Meerow et al., 2016;Moser et al., 2019;Righi et al., 2015;Serfilippi and Ramnath, 2018) or focused on the methodological aspects of types of resilience measurement. For example, Schipper and Langston (2015) explored the theoretical and practical gap between resilience and resilience measurement, and Asadzadeh et al. (2017) focused specifically on developing composite indices.
Little research has been carried out on the interconnected nature of hazards and how this is currently captured in resilience measurement approaches or how resilience approaches are supporting decision-making and resilience implementation (de Angeli et al., 2022;Mehryar et al., 2022). This paper undertakes a critical review by evaluating the persistent gaps among available services and how filling these gaps can support implementation and decision making. We aim to assess and reflect on a set of the most comprehensively and widely applied resilience measurement approaches from the literature. By diving deeper into the challenges and limitations of these approaches, as well as the opportunities each of them provide for assessing and building resilience, we learn more about the persistent gaps so that sound decisions can be taken to fill them. We recognize that not all approaches to resilience measurement can be universally generalized and compared across different socio-ecological, political, and economic contexts: we aim to provide an overview of different approaches to resilience measurement and their key characteristics, rather than to conduct a comparative study.
To do this, we focus particularly on three research questions: 1. Are the compounding socioeconomic and climatic risk drivers being considered in resilience measurement approaches, and, if so, how? 2. What are the methodological and technical challenges associated with these approaches and the opportunities they offer? 3. What are the challenges and opportunities to application of these approaches and how are they supporting decision-making processes?
While the first and second questions are about the content of approaches (measures used, data collection and analysis process, target, etc.), the third question is about the application and decision-support aspects of these approaches.
Based on a critical review, we identified an in-depth review of 17 climate and disaster resilience measurement approaches (developed by research, government, and private and public organizations). This paper aims to offer critical insights into how existing resilience measurement approaches may engender systems thinking. Additionally, we apply an interpretative analysis and critical evaluation to reflect on the strengths and weaknesses of these measurement approaches, their inherent contradictions and similarities, and how they can address the uncertainty and complexity caused by extreme events by managing and promoting resilience across different contexts. Finally, this critical review is also meant to guide future developments in developing approaches to climate and disaster resilience assessment and foster not only an enhanced understanding of how the nature of systemic risks can be measured but also a better standardization of approaches to resilience measurement. We do not aim to compare the tools as such, but rather to outline their characteristics, which will represent opportunities and in different contexts in future developments. Ideally, the next generation of resilience measurement applications will be better able to consider multiple vulnerabilities and risks, both conceptually and practically, as well as to learn from other decision-making and communication strategies across a broader context of these 17 approaches to climate and disaster resilience measurement. This paper is organized as follows. First, we review the literature on three dimensions relevant to evaluating approaches to climate and disaster resilience measurement: 1) the underlying concepts of measuring risk and resilience and its interactions between physical and social risk drivers; 2) methodological and technical aspects of measuring resilience; and 3) application and practicability of resilience measurement for decision support. Second, we describe the selection criteria of approaches to resilience measurement and the assessment criteria used for the review (Section 3). Third, we present findings and discusses potentials of the review (Section 4). Finally, we provide recommendations and conclusions based on our findings (Section 5).

Concepts of resilience and multiple vulnerabilities and risk drivers
In its first applications to ecological and engineering contexts, the concept of resilience was associated with the notion of "bouncing back" to the pre-impact state based on system equilibrium and recovery. In the climate and disaster resilience literature, however, researchers and practitioners have increasingly stressed forward-looking conceptualizations, such as "building back better" or "bouncing forward" (Hynes et al., 2020;Keating et al., 2016;Manyena et al., 2011;UNISDR, 2017). The concept of resilience has thus evolved from being focused on specific dimensions, such as ecological systems (Holling, 1973), to an integrated and holistic vision encompassing multiple disciplines and embracing the diversity of contexts, purposes, scales, and systems that these bring (Adger, 2000;Aldunce et al., 2015;Baggio et al., 2015;Moser et al., 2019).
While the concept of resilience is contested and considers a variety of definitions (Olsson et al., 2015;Parker, 2020), in the context of climate and disaster resilience it can be broadly defined as the "ability of a system, community or society to pursue its social, ecological and economic development objectives, while managing its disaster risks over time in a mutually reinforcing way" (Keating et al. 2016, p. 80). Regarding this notion, the multiple capital approach was popularized by the Sustainable Livelihoods Framework (DfID, 1999). By exploring resilience in this way, holistic links between the physical, financial, human, social, and natural become more accessible. This approach takes into account the multiple climatic and socioeconomic factors that contribute to individual and collective resilience, such as managed and unmanaged landscapes, financial resources, social networks, communications, governance, and different worldviews.
A crucial concept related to climate and disaster resilience is the concept of risk which is broadly defined as the probability of suffering harm or loss. The IPCC glossary defines risk as the interaction between hazard (triggered by an event or trend related to climate change), vulnerability (susceptibility to harm), and exposure (people, assets, or ecosystems potentially facing damages). It can be expressed as the probability of an occurrence at a given location (IPCC, 2019). In other words, approaches to risk identify vulnerabilities in a system and propose action to withstand hazards to an acceptable level (Linkov et al., 2014). For risk approaches, reducing risks thus means taking action that reduces either the probability of an event happening or its negative impacts. Such actions, including the management of residual risks, are defined as risk management (UNDRR, 2017b). Many of these approaches focus on a single hazard as a flood, drought, or earthquake at a specific location or affecting a specific type of system element (road infrastructure; energy system etc.); and they are often based on a cost-benefit assumption (see Mechler et al., 2014).
While this may be sufficient for short-term assessments, a comprehensive system understanding of the societal functions and capacities that can enable and support risk management activities before and after an event (i.e., resilience capacities) is not usually considered. With increasingly complex, non-linear systems and uncertainties at play in our societies today and in the future, approaches to resilience measurement play a critical role in risk management. They can thus further complement and improve risk management approaches as a positive framework that integrates risk drivers and coping, adaptive, and transformative capacities (Linkov et al., 2014;Mochizuki et al., 2018).
A key aspect thus needing attention is that neither the interrelations among sub-systems within a system (physical and societal) nor the systems dynamics have been sufficiently understood (de Bruijn et al. 2017). Many regions of the world are prone to multiple risks, however, and can thus experience multiple hazards simultaneously (Gill and Malamud, 2014;Raymond et al., 2020;

Table 1
Presents some of the underlying terminologies used for risk and resilience.
Multi-hazard or multivariate event refers to a disaster arising from more than one environmental driver) that has the potential to trigger damage and loss to the same exposed system (location, community, and its functions (infrastructure, service provision etc.) While there is debate regarding the terminology employed to define multi-, compound, and interconnected risks (Cutter, 2018;Pescaroli and Alexander, 2018;Zscheischler et al., 2018), generally speaking, multi-hazard events occurring in a temporal sequence will consider cascade effects. A specific type of multiple hazards is a compound hazard where hazards occur concurrently or sequentially. The concept of interconnected social (-economic) risks considers the underlying drivers of risks in terms of their social, economic, and human interactions. The social construction of risks implies that vulnerable groups (such as children, older people, poorer societies, or disabled people) at local places, or specific social groupings are particularly susceptible to climate and disaster risks (World Bank, 2010). Interconnections between multiple social-economic risk drivers increase vulnerability and reduce the adaptive capacity of resilience. The different degree of groups against the various hazards which may change in may change with time as consequence of different factor is defined multi vulnerability (Gallina et al., 2016). The analysis of both physical and social risks can be referred to as systemic risks, where the dynamics of social-ecological systems can cause failures of one part of the system, resulting in a cascading failure of other parts (Renn, 2021;Renn et al., 2017). For example, "societal mechanisms" can tie together the risks from multiple physical events, whether these are due to resource constraints or health considerations (such as power outages). A better understanding and management of the causes and consequences of systemic risks is essential for decision makers to prepare for future challenges . 2020). In this context, biophysical hazard assessments and models have emphasized the need for analyzing multi-hazard risks (Kappes et al., 2012;Tilloy et al., 2019). For example, Wahl et al. (2015) and Zscheischler et al. (2020) showed latent and pre-existing natural conditions of climatic forces (like a combination of saturated soils and precipitation) that intensify or cause hazard events (like floods or droughts). This raises questions about managing risks across consecutive events  and resilience strategies related to compound events Ward et al., 2020). For example, the growing understanding of complex interactions between flood and drought risks and the underlying social drivers of vulnerable groups and critical infrastructures increase the need to consider systemic risks to strengthen resilience capacity and the strategies of all involved.
Differentiating between multi-hazard events, compound and interconnected social risks, and systemic risks serves not only to better identify appropriate measures for sustainable development goals and disaster risk reduction strategies (see Table 1), but it may also help to improve the understanding of resilience to physical and social-economic risk drivers (Pescaroli and Alexander, 2018;Scolobig et al., 2017). But also, the integration different exposed (sub)systems or different vulnerabilities of the same (sub-)system according to multiple hazards needs more attention. Generally, two types of multi risk approaches are considered. While multi-hazard risk assessments take vulnerability as independent from a specific hazard, multi-risk assessments include assessing multi-hazards and multivulnerability concepts.
The importance of managing the occurrence of physical events and social risks and the interactions between them has been discussed conceptually in the physical science-based research context (see, for example, Zscheischler et al. 2020;Wahl et al. 2015) and some progress has been implemented in practice by using system-thinking approaches, and comprehensive risk reduction and resilience approaches (Barrett and Constas, 2014;Depietri, 2020;Manyena, 2006). For example, Raymond et al. (2020) stressed the importance of considering social activities that can directly affect the severity of events. They note that social forces like governance, markets, and other decision-making structures in combination with population exposure and vulnerability can create additional interconnected social-economic risks in the course of physical events, like flooding or heatwaves. While approaches to climate and disaster resilience measurement have advanced in this regard by placing social systems at the center of attention, they are still largely lacking practicability in the context of systemic risks (Shi et al., 2020). These scholars identify that the climatic and social-economic risks that underlie resilience are not mutually exclusive and often overlap. To better implement resilience measurement concepts in practice, it will be critical to advance risk management and decision-support methods through the use of structured and systematic data collection and analysis techniques and also through the integration of inclusive and participatory approaches so that the systemic nature of risks can be measured.

Methodological and technical aspects
In practice, Twigg (2009) and Bahadur et al. (2013) found that a set of quantifiable indicators was the most common methodology used. The development and operationalization of such approaches does, however, require a comprehensive understanding of the methodology and techniques involved. In terms of the distinction between approaches to resilience measurement, a meta-analysis of peer-reviewed articles revealed that resilience approaches can be defined and measured as a system trait, process, or outcome (Moser et al. 2019). Approaches to resilience measurement can thus focus on identifying system characteristics (system traits), actions, and interventions (process), or temporary states of a system (outcome).
Furthermore, existing reviews on the technical aspects of approaches to resilience (e.g., Cutter et al. (2016) and Asadzadeh et al. (2017)) categorized the indicator-based approaches into four main types: scorecards, indices, models, and toolkits. Indices and scorecard approaches can be broadly defined as indicator-based approaches and are used to illustrate the multiple dimensions of resilience; but they ultimately aim to reduce complexity by assigning numerical values to observable constructs. While (composite) indices focus on the aggregation of multi-dimensional indicators into one weighted index using statistical methods, the scorecard type provides an evaluation of the progress toward a defined or previously measured benchmark. In contrast, models propose a theoretical background for quantitative indicators using a deductive process, to capture approximations of the real world. They often follow a hypothesis-based mechanism for measuring resilience through data, models, or specific procedures. Toolkits (or tool-based approaches) have a broader scope and develop processes for assessing resilience using one or a combination of the approaches mentioned before (Cutter et al. 2016).
A crucial component of any climate and disaster resilience measurement is the methodological approach it uses for evaluating and quantifying resilience (Cutter, 2016;Nardo et al., 2005). Despite the need for technical standardization, resilience is specific to a diversity of contexts determined by time, space, livelihood types, and shocks, which narrows the use of generic indicators across different contexts. The required context-specificity has resulted in a variety of technical methods, ranging from household surveys, to key informant interviews, to group discussions (Campbell et al., 2019;Norris et al., 2008;Sherrieb et al., 2010). For this, the use of multiple methods and data collection techniques in a structured and systematic manner over time is essential (Keating et al., 2017b). However, the balance between standardized methodologies and the integration of context-specific aspects remains a major challenge for any resilience measurement developer. Laurien and Keating (2019) illustrated this challenge by assessing resilience indicators through quantitative methods and qualitative methods. The approach emphasized that a hybrid approach to collecting and analyzing qualitative and quantitative data can increase the quality of resilience measurement approaches significantly.

Aspects of decision support
Recent innovations regarding approaches to resilience measurement have concentrated on methodologies, while a less explored aspect of resilience measurement is its applicability and practicability for end-users. This is garnering further attention, as there has been a recent surge in the application of resilience measurement tools and indices across various fields, particularly by civil societies and practitioners charged with implementing resilience-based strategies "on the ground" (Jones et al., 2018;Moser et al., 2019). The complexity and time-intensive approaches for measuring multiple dimensions of resilience has, despite these challenges, led to improved systems-thinking and public engagement capacities for practitioners and has holistically supported resilience activity planning (Keating and Hanger-Kopp, 2020). Douxchamps et al. (2017) conclude that a major challenge in measuring resilience is to ensure that the means of measurement are operational in practice and reduce the complexity of resilience. The authors emphasize that measurement approaches should comprise both quantitative and qualitative data and include systemic indicators as well as indicators of stability and shocks over time. Similarly, Jones et al. (2018) distinguish between subjective and objective approaches to resilience measurement to account for different measurable applications and user needs. Some advanced resilience approaches require a significant amount of data and resources before a quality assessment is undertaken to produce standardized results, while other approaches emphasize the role of end-users, like households and policymakers who have a perception of the underlying resilience.
Overall, these reviews highlight the research-practice-policy gap. For decision support, differentiating between hazard-specific or broader climate-related and shock-related measurements is critical and needs to be addressed in the initial stage of the measurement approach.  (Carpenter et al., 2001). Additional dimensions like the application context, customizability, ownerships of the process, and adaptability of the action planning resulting from the measurement outputs should, however, be considered as well.

Selection of resilience measurement approaches
The number of approaches to the measurement climate and disaster resilience has constantly increased. Based on a literature search and expert identification, 17 representative approaches to climate and disaster resilience measurement are selected for the critical analysis of this study (Table 2). For the literature search, we build on three steps. First, we used standard platforms (e.g., Web of Science, Scopus, Google Scholar) to collect both scientific and non-scientific documents. A search string combining "climate/hazard/ disaster" AND "resilience" AND "measurement/assessment" AND "tool/toolkit/index/matrix/indicator" were used. Secondly, we identified relevant approaches from this dataset (see also Mehryar et al., 2022). Thirdly, we consolidate our dataset by including practitioners and experts working with community resilience tools and approaches. The expert identification process collected experts' experience on the approach to resilience measurement from professionals in academia, NGOs, and the private sector.
The final selection of the representative approaches in the climate and disaster resilience field is based on: 1. The approaches, including indicators, indices, and tools developed and used to assess resilience in a holistic manner, considering physical risks and multiple connected social drivers. 2. The measurement approach being applied beyond pilot studies to support decision making and policymaking. 3. The measurement approach that has been developed and implemented by different developers, operators, and users (e.g., researchers, NGOs, businesses, and public bodies) across different geographic and societal scales to reflect an audience of diverse user needs.
As with any critical evaluation method, objectivity is impossible to achieve. The selection is constrained by the selection process and the available materials. To include a variety of different approaches, the selection (as shown in Table 2) is done using Scopus and Google Scholar, then sense-checked with experts from the Zurich Flood Resilience Alliance, including experts from academia, NGOs, and the private sector. The approaches identified are then analyzed and reviewed by the three researchers (whose backgrounds are in social, economic, and natural science). The process in this study closely follows the guidelines of methods for critical literature reviews (Paré et al., 2015). To minimize subjective judgment in the review process, an intercoder reliability method is used when analyzing the content of each measurement approach (Lavrakas 2008). The intercoder reliability method describes the agreement between different researchers regarding how the same data should be coded (see Appendix A for more details) (O'Connor and Joffe, 2020).

Assessment criteria
Due to the wealth of existing resilience measurement approaches, selecting appropriate approaches for a given economic, developmental, or socioecological context can be challenging. To be able to provide an overview of the characteristics and scope of each selected measurement approach, we identified the following assessment criteria from three domains: 1 Multiple risk drivers in resilience measurement approaches 1 The authors note that some technical aspects are subject to a longer evaluation process due to the different interpretations and background of the evaluators. In this case the evaluation of the intercoder reliability was discussed in more detail and additional practitioner perspectives were included.
• Hazard type: What hazard types can be considered in the assessment? e.g., specific (flood or droughts) or climate-related multihazards (more than one hazard but no interactions). It is also possible for the approach to consider general shocks (no focus on a specific hazard). • Social risks addressed: Is there a focus on the dependencies of social risks in the resilience measurement approach? Here it is possible to focus on specific thematic areas; development in a wider context or social risks are more generically considered. • Compound risk addressed: Is it possible to account for interactions between multiple hazards and social risks? Interactions can be considered by different levels; hence the reviewer team can find a common agreement by responding Yes; Yes, but general; or No focus.

Methodological and technical aspects
• Disaster phase: During disaster = The tool allows resilience to be measured during a disaster; ex ante = The tool allows resilience to be measure in the initial state before a disaster; ex post = The tool allows the measurement of subsequent states/trajectories after a disaster • Technical complexity: User-friendly = The tool is accessible to non-experts, and is unambiguous and easy to interpret without specific software or equipment being required; Moderate = The tool requires expert knowledge and technical expertise to be run OR specific software or equipment; Complex = The tool requires expert knowledge and technical expertise to be run AND specific software or equipment. • Data type: Data collection can be conducted through qualitative or quantitative data or a combination of the two.
• Comparability: Fully comparable = The tool provides resilience measurements that are fully comparable AND usable together with other tools' results; Moderately comparable = The tool provides resilience measurements that are comparable OR usable together with other tools' results; Not comparable = The tool provides resilience measurements that are neither comparable nor usable with other tools' results • Holistic framework: Number and type of resilience dimensions (capital) covered including "natural," social," "human," "financial," and "physical." Note that the original approach sometimes uses similar terms, like ecological for natural. • Duration of assessment: How long does it take to carry out the measurement? Long = Over 4 months; Moderate = Between 1 and 4 months; Short = Less than a month; Not specified = no information available and reviewers find it difficult to evaluate. Decision support • Customizability: Not customizable = The measurement approach requires the same indicators to be measured in all applications; Partially customizable = The approach allows users some flexibility as to which indicators are included and how they are specified; Fully customizable = The approach does not use specific indicators, allowing for full flexibility in the choice of indicators by users • Unit of application: What is the unit of application? Household, community (city), county, grid (km 2 ) or country. • Open access: Is the approach or method openly available and free to use: Open = The tool and its instructions are publicly available for the users; Regulated access = Either the tool or the instructions are not open-access; Restricted access = The tool is not publicly available for users. • Ownership: User's ownership = Users implement the tool and have access to data and results independently; Developers'/implementors' ownership = Developers implement the tool and have access to the data and results; Joint ownership = The tool is implemented via a collaboration between the developers/implementors and users, and both parties have access to the data and results. • Action planning: Do the tools include a post-resilience measurement process which supports decision making and action planning?

Findings and discussions
Based on the review of resilience measurement that was undertaken, we now distill from it the following findings in terms of multiple hazard dimensions, applications, methodological, and technical aspects, as visualized in Table 3.

Theme 1: Multiple risk drivers in approaches to resilience measurement
Risk assessment in resilience measurement. Assessing risk and drivers of risks is an important step in evaluating resilience, particularly if there are multiple risk drivers. Risk assessment, however, has rarely been included in resilience assessment approaches: risk and resilience approaches are thus often being assessed separately using different approaches to measuring resilience-either by distinct methods and measurement approaches being created for risk or resilience assessment, or by one approach being used to address each one separately. Among the 17 approaches analyzed, there are a few examples (i.e., RIT, ICLEI, and DRSC) in which the latter case is used, with risk assessments often being carried out prior to resilience assessments to identify and prioritize (multiple) shocks and stresses. This helps communities to define the depth and extent of the analysis required for each individual hazard (see RIT, ICLEI, and DRSC). Integrating a comprehensive risk assessment (i.e., including assessment of current and future hazards, exposure, and vulnerability) at the beginning of the resilience measurement is observed as being better at communicating resilience strengths and weaknesses of communities, particularly where multiple risks are in play.
Measuring multiple hazard events. None of the 17 approaches considered multi-hazard events in their indicator design, although some developers did consider a multiple hazard applicability. Most of the approaches to resilience measurement that we reviewed were designed to measure resilience to a single hazard (e.g., FRMC, REA), have generic indicators that are used for measuring resilience against a variety of risks, with the downside of being very broad (e.g., RIMA-II, STRESS, IEF), or have hazard-specific indicators that can be selected and measured for each hazard independently. For the latter case some approaches such as DRSC, ICLEI, and NaHRSI have a process of analyzing and identifying the most frequent and most severe hazards in each case study, which helps communities to specify the indicator's responses. Our results thus show the lack of approaches currently capable of adopting a systems perspective by accounting for the interactions of multiple hazards and their social risk drivers.
Compound risks. Some approaches do allow for generic multiple hazards to be accounted for, but this has only been possible where the indicators included are non-hazard-specific (e.g., ARC-D, BRIC). Neither of these approaches, however, support the measuring of resilience against risks linked to compound hazards (i.e., when multiple events occur simultaneously and their impacts overlap). There is thus a need for measurement frameworks that better address compound events and the interrelations between hazards, including their simultaneous or cumulative occurrence and potential interactions. It is also important to consider, as Carpenter et al. (2001) emphasize, that resilience measurement approaches with a very general focus are inevitably unable to consider compound risks, unless they consider the "resilience to what" question. Some resilience measurement approaches to generic shocks can, however, be used to develop a model that supports the understanding of multiple hazards and cascading effects.
Socioeconomic risk drivers. The underlying risk drivers from the social dimension are widely taken into account in the conceptualization of resilience. Many developers applying the measurement approach in highly vulnerable geographical regions select the indicator based on socioeconomic drivers. Socioeconomic indicators are integrated into the design phase of such approaches (e.g., by using multiple financial, human, and social capitals) and into the regional selection; the implementation of interconnected socialeconomic indicators, however, and what they mean for resilience remain largely unsolved. Scorecards and toolkits may emphasize the use of social indicators for climatic risks, but they often become generic when actually applied. For example, we have not identified any socioeconomic indicators that are specifically gender-and resilience-based. Furthermore, index-based approaches largely rely on the availability of data, highlighting the benefit of the data collection process being integrated into the framework design. Our results also show shortcomings in the integration of gender, political, and governance resilience data.

Theme 2: Methodological and technical aspects of resilience measurement
Technical complexity. As resilience is a multi-faceted concept, it is not surprising that most of the approaches to resilience measurement entail technically complex and time-consuming data collection and assessment processes. As a result, many measurement approaches that have been applied in a large number of countries and communities use a combination of qualitative and quantitative data and methods, which helps to balance the level of complexity. Technical complexity remains a key challenge for applicability and the user-level ownership of the approach. Without technical support and sufficient funding, scaling effects and further policy impact was not found.
Holistic frameworks. Most of the measurement approaches that we reviewed are based on measuring capacities related to the different dimensions of resilience. In an effort to holistically cover the various dimensions of resilience, nearly all measurement approaches use frameworks that are identical or closely resembling the 5 capitals framework (DfID, 1999), which seems to have strongly shaped the resilience measurement landscape. Additionally, coping, absorptive, and adaptive capacity are frequently considered in both the definition and the design of a reviewed framework, while transformative capacities are less represented. For instance, Béné et al.'s (2012) three resilience capacities were explicitly mentioned in the FRMC, ARC-D and STRESS, yet indicators mainly reflected absorptive capacity (e.g., number of shelters), and transformative capacity are less represented.
Among the 5 capitals, natural capital or ecological capital are the least represented in the approaches reviewed, and there was a lack of indicators specific to ecological systems and ecosystem services. While broader approaches such as DRSC, FRMC, and ARC-D peripherally include ecological considerations, such as the presence or absence of protected areas, the link between healthy ecosystems and resilience is rarely addressed specifically (e.g., by including biodiversity metrics in the measurement approaches). Among the measurement approaches reviewed, two present a specific focus on natural capital (ATLAS and IEF). These take a slightly different approach to resilience as they are based on the principle that resilient ecosystems (e.g., through species biodiversity or intact ecosystem dynamics) are more resistant to shocks and stresses, and thus essential for resilient societies and communities. An explanation for the shallow consideration of natural capital in many approaches could be the complexity of natural system dynamics and the uncertainties of interactions to social systems. A further explanation may be that the role of ecological systems in influencing disaster resilience may not yet be sufficiently embedded in a response-focused perspective on disasters.
Tool methodology. The approaches we reviewed represent different methods of measuring and quantifying resilience, often depending on the scale for which the approach is designed. Indices and scorecards are heavily used for higher-level assessment, whereas toolkits based on focus-group discussions and expert involvement have been used for community and household-level assessments. Validation concerns and the extent to which the findings accurately contribute to resilience-building efforts could be not evaluated consistently because of the absence details in most approaches to dealing with sensitivity and validity issues and contradictions in the meaning and definition of what validation is. More mainstreaming is needed to align the efforts in measuring resilience in a consistent and valid way. While data reliability and quality issues have been considered in a variety of approaches (see FRMC; Campbell (2019)), the use of mixed methods to increase data robustness is rarely considered by other approaches. It is important to note here that approaches to resilience measurement are constantly evolving, and new approaches are regularly developed and adapted over time. For example, while the traditional focus has so far largely been on "objective," and purely quantitative approaches to measuring resilience have been favored (e.g., RIMA, BRIC), there has been a renewed emphasis on "subjective" resilience measurements, which are thought to capture its social and human dimensions more comprehensively (RRR, STRESS) (Cutter et al., 2014;2010;Carpenter et al., 2001). Accordingly, the less tangible aspects of resilience, such as social networks and governance, have recently been in the spotlight in approaches to resilience measurement. By conducting a comparison between RIMA-II and the RRR framework, Jones and D'Errico (2019) highlighted the need for hybrid approaches.
Disaster cycle. Most approaches are designed for measuring resilience ex ante, rather than allowing the measurement of subsequent resilience states or trajectories after a disaster (e.g., FRMC, ATLAS). This is of paramount importance for effective decision making, which ideally requires resilience measurements at different points in time across the disaster cycle to be adaptive. Likewise, only few approaches provided the possibility of measuring resilience during or quickly after a disaster. The FRMC approach developed a comprehensive ex-post study for comprehensively measuring impacts in addition to ex-ante resilience characteristics, while the RRR is able to conduct mobile surveys during or after a disaster. The challenge of providing instant results is most likely due to most measurements being too lengthy (see technical complexity).

Theme 3: Decision-support
Customizability and global practicability. Most of the tools have a flexible application approach, meaning that users can identify the set of indicators/questions/indices which are most relevant for their problems/objectives and thus need to be included in their analysis. This is particularly important for global multi-hazard assessments, where many different indicators for various types of hazards exist and their importance and level of relevance may vary for different contexts. For example, both DRSC and ICLEI allow users to select indicators that are applicable to their urban area contexts through a preliminary stakeholder workshop. The downside of this approach could be that the holistic and multi-system aspects of measuring resilience can be lost if users focus only on specific indicators and elements rather than assessing resilience as a whole. Approaches with a context-specific focus can, however, be flexible in terms of using available information while also prioritizing context-specific questions (e.g., ICLEI governance issues in urban regions or RIMIA-II for food security) over generic ones (or one-size-fits-all).
Unit of application for decision support. Most of the approaches analyzed have a specific unit of application such as city, community, or household level, whereas a few (i.e., CDRI, ANDRI, and NaHRSI) have no unit of application, meaning these tools and indicators are developed in a way that can be implemented and applied at various levels. While the second approach provides more flexibility in application terms, the first approach provides more context-relevant indicators. For example, approaches developed specifically for measuring urban resilience (e.g., ICLEI, DRSC, CRPT, RIT, and CDRI) have a stronger focus on integrating climate change adaptation and DRM strategies into city development planning. This is based on the prevalent view that urban resilience means integrating climate change and disaster risk reduction strategies into the key development priorities in urban areas. Such tools, for example, have a particular focus on evaluating the suitability of spatial planning/zoning, building codes, urban development projects, plans and policies, and roles and responsibilities of urban-related sectors (e.g., urban planning, urban landscape, architecture, real estate, and construction) for climate change adaptation and disaster risk reduction. This is technically done in two ways: 1) by incorporating specific indicators/questions to assess the quality of urban planning and governance systems; and 2) after completion of resilience measurement by analyzing the alignment of recommended interventions with the existing visions, plans, and programs of the cities. See, for example, the action planning stages of DRSC, CRF, ICLEI, and RIT. In the more general approaches, such aspects related to urban planning and governance are usually missed or less recognized.
Sectoral silos. We note that approaches to resilience measurement are rarely developed for, or tailored to, the private sector and businesses (Cutter et al., 2016). This highlights the inherent issue of sectoral silos and the lack of incentives for the private sector to fund resilience programs. Additionally, as illustrated by tools originating from the natural sciences sector (e.g., Raymond et al., 2017), due to the broad application field of resilience, it appears that different research communities and sectors are measuring resilience using the same concepts, yet different terminologies.
Implementation plan and program. Supporting decision making and encouraging resilience building goes beyond measuring resilience. While resilience measurement can be an important step in initiating resilience thinking, resilience acting requires consideration of governance aspects such as power dynamics, availability of resources such as long-term funding (Keating and Hanger-Kopp, 2020), suitable policies, and relevant roles and responsibilities. Some tools such as STRESS, RRR, and DRSC have intervention design and prioritization as a part of their overall approach (i.e., following the resilience measurement). Some other approaches, such as RIT and ICLEI, go further and provide a short-and long-term resilience plan and program that support the materialization of resilience interventions, as well as ensuring long-term resilience planning rather than simply focusing on single isolated solutions. Such approaches generate a complete and detailed roadmap of different stakeholders and responsibilities, institutional mechanism, technical support, policy changes, or the fundraising needed to implement the resilience strategies.
Policy and communication goals. While assessing the links between measurement and policy goals is not the primary goal of this study, overlaps between the reviewed tools and the United Nations Sendai Framework for Disaster Risk Reduction (2015) objectives and indicators are identified. DRSC and ARC-D, for instance, assist local governments in monitoring and reviewing their progress and challenges in the implementation of the Sendai Framework for Disaster Risk Reduction: 2015-2030. They are, therefore, structured around UNDRR's ten essentials for making cities resilient. The BRIC framework is also prevalent throughout the reviewed tools. Nevertheless, in order to achieve global goals and targets, such as the Sustainable Development Goals (SDGs), better mapping and mainstreaming of resilience indicators onto global targets is required.

Conclusions
In response to the demand for information by the policy, practice, and research communities, there has been a surge in fresh approaches to climate and disaster resilience measurement. This paper conducted a review and an assessment to identify the gaps in approaches to resilience measurement related to systemic risks-particularly in multi-hazard environments-with a view to supporting decision making in various contexts. The purpose of this research was not to identify and compare best practices across resilience measurement approaches operating in different contexts, but rather to provide an overview of existing approaches and their characteristics. This review is also meant to guide the further development of resilience assessment methods and the approaches they take. Seventeen approaches to resilience measurement developed by research, government, and private and civil society organizations were selected and evaluated according to a set of assessment criteria ranging from the flexibility of the measurement approach to their breadth and complexity.
Based on these findings we propose a new generation of approaches to systemic resilience measurement that is able to measure how social and climatic risks are interconnected and provide reliable and useful information for practitioners. This new approach, where multiple hazards are taken into account, needs to be context-specific; it also needs to account for multiple underlying social risk drivers and address compound events, while decision-making strategies should consider multiple vulnerabilities. To overcome the gaps identified in this paper and to resist the temptation to create individual strategies for different hazards, the new generation of measurement approaches needs clear standards and definitions in the measurement methodology so as to avoid inconsistencies and poor data quality. As an organizing concept for capturing complexity and uncertainty within and between dynamic systems, the resilience measurement framework should further strive to combine both process and outcome perspectives to represent resilience in the most holistic and systems-thinking manner possible. Tensions may exist between top-down measurement and participatory approaches; however, co-creation could become a critical factor to explicitly recognize resilience measurement approaches as an essential risk management approach. For example, ARC-D and the FRMC are designed to be run with local communities using participatory approaches, thereby allowing the co-creation of resilience building measures. There is great potential for top-down created Earth observation data to be further integrated into resilience data collection, as well as mapping of resilience measurement approach results, for instance, through participatory mapping workshops (using citizen science). As the RIT document explains, resilience strategy workshops are invaluable in successfully aligning interests and uniting people from various hazard management sectors around a common goal. Framework development should focus on multiple functionalities and interconnected networks (not only hazard interactions but also social functions). This leads to some methodological requirements, including hazard-specific and context-specific indicators, and it also requires the specific consideration of methods to assess interactions among hazards and how these affect development. Methods that focus on the temporal dynamics of resilience while reducing data collection efforts in the field and are desperately needed. Measuring resilience multiple times in the same location can significantly enhance our understanding of social and physical interactions during and after a disaster and allows more rapid and target-orientated decisions. As discussed, a given location can be exposed to multiple hazards, which can sometimes be temporally interrelated. Although most resilience measurement approaches are designed to account for a single hazard at a time, prioritizing the risks originating from multiple and compound events is critical to inform appropriate mitigation or preparedness measures (Zaidi, 2018). This gap is most likely due to the fact that comparability between resilience measurement approaches associated with different types of natural hazard is hampered by the different methodologies and metrics used for different disaster types (Marzocchi et al., 2012).
In this critical review, we identified that although social factors are generally considered, more effort is necessary to improve the understanding of interactions between social factors and climatic risks and how important such understanding is for enhancing resilience capacities. For instance, the approaches reviewed are detailed in terms of measuring climate-related hazards or generic shocks. However, all approaches fall short on measuring interactions between physical and social risks including systemic risks. While this gap has been discussed conceptually in the literature for multi-risk assessments (de Angeli et al., 2022) it also needs to be considered for future developments in resilience measurements, both in terms of conceptual aspects and technical implementations. The fact that compound risks of social and physical risks are a clearly underestimated field in the interactions between social factors has been already identified by other scholars in an empirical resilience assessment (e.g., Hochrainer-Stigler et al. (2021)).
By enhancing resilience measurement approaches to compound risks, the implementation of single hazard solutions grounded in maladaptation and silo-thinking can be reduced considering that practitioners and policymakers are integrating them in their programs and decision-making processes. In addition, the co-benefits of implementing compound risk assessment as an integrative component at an early stage of resilience measurement will favor long-term planning for resilience. Successful resilience strategies address multiple vulnerabilities and deliver benefits across a wide range of hazards and climate impacts. Achieving the co-benefits of resilience strategies and intervention for different types of hazards is acritical motive for developing multi-hazard approaches. For example, the three dividends of resilience (developing co-benefits, or uses, of a specific disaster risk management investment (see, Surminski et al. (2016)) is currently under-represented in resilience measurement approaches. The consideration of multiple resilience dimensions can help to integrate holistic thinking into decision making. Strengthening the integration of measures relating to cobenefits, such as natural capital or cultural services, will be critical to demonstrate the wider benefits of resilient solutions, such as nature-based solutions.
Likewise, the balance between data collection requirements and applicability remains a key challenge. This is because lighter approaches usually fail to comprehensively address the underlying challenge (to consider the right balance between addressing resilience as an outcome or a process), for example, to be tailored or broad, to be subjective or objective, complex or oversimplified, rapid or too lengthy. We also recognize that there is no "one-size-fits-all" solution and that every approach has benefits and weaknesses in different contexts and for different purposes. This needs to be acknowledged in the methodology and design of approaches to support decision making and in order to achieve common ownership and relevant action planning. For example, multi-hazard resilience building requires interventions that cross departmental barriers. As the RIT document explains, resilience strategy workshops are invaluable in successfully aligning interests and uniting people from various hazard management sectors around a common goal.
Resilience measurement approaches are constantly evolving, and new approaches are regularly being developed and adapted. This paper expresses our belief that the next generation of resilience measurement approaches will likely consider a more dynamic system that can integrate compound risks and multiple hazards over time at the center of a framework: a measurement approach that informs decision-making strategies with advanced data techniques and methods for considering multiple vulnerabilities, while including more details on equity and justice principles.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.