1 Introduction

As future cities become larger and ever more densely populated the impacts of natural disasters, such as earthquakes, become more severe. In an attempt to reduce the impact that such disasters have on communities the United Nations adopted the Sendai Framework for Disaster Risk Reduction (UNDRR 2015) as a voluntary, non-binding instrument to drive a “substantial reduction in disaster risk and losses in lives, livelihoods and health and in the economic, physical, social, cultural and environmental assets of persons, businesses, communities and countries.”. The Sendai Framework replaced the Hyogo Framework for Action (UNISDR 2005) and although the principle objectives of Hyogo and Sendai are the same, the means for achieving the objectives differ. Whilst the Hyogo Framework acknowledged the need to manage disaster risk, its primary focus was on managing the disaster event. In contrast, the Sendai Framework has a much clearer focus on managing disaster risk, identifying four key priority areas for action over a 15-year period from 2015 to 2030 (UNDRR 2015). In particular, the Sendai Framework identified the need for:

  1. 1.

    a greater understanding of disaster risk in all its dimensions (exposure, vulnerability and hazard characteristics);

  2. 2.

    a strengthening of disaster risk governance at all levels to better manage disaster risk including clear vision, plans, competences, guidance and coordination within and across sectors;

  3. 3.

    investment in disaster risk reduction for resilience, including structural and nonstructural mitigations to enhance disaster preparedness and recovery; and

  4. 4.

    a commitment to build-back-better in recovery, rehabilitation and reconstruction.

In addition, the Sendai Framework recognised that, whilst Governments have the primary responsibility to reduce disaster risk, other non-governmental stakeholders such as the private sector, community organisations and businesses can influence disaster risk response or planning.

To support the application of the Sendai Framework the UN commissioned the Disaster Resilience Scorecard for Cities (UNISDR 2017). It with the Disaster Resilience Scorecard for Cities (ibid) scores community resilience to disaster events by considering resilience factors against 10 Essentials (organise for disaster resilience; identify, understand and use current and future risk scenarios; strengthen financial capability for resilience; pursue resilient urban development and design; safeguard natural buffers to enhance the protective functions offered by natural capital; strengthen institutional capacity for resilience; understand and strengthen societal capacity for resilience; increase infrastructure resilience; ensure effective disaster response; expedite recovery and build back better) (UNISDR 2017). From the experience of the local Governments that have used the Scorecard it has been applied at two levels: preliminary scoping level; or detailed assessment level. Whilst a detailed evaluation of the lessons learnt from the application of the Scorecard (Schofield and Twigg 2019) identified its success in providing a practical framing of disaster risk reduction issues, it also identified the need to contextualise the Scorecard to reflect local hazard characteristics and develop a common understanding of the issues in terms of both local resilience and disaster risk. The Disaster Resilience Scorecard for Cities provided (ibid) provided the theoretical basis for assessing community resilience to earthquake induced liquefaction in the LIQUEFACT project.

“Earthquake-induced soil liquefaction phenomena result from the interaction of soil particles and porewater under the shear stress and shear strain reversals induced by earthquake shaking” (National Academy of Sciences 2016). The extensive soil liquefaction that occurred in recent earthquakes such as the 2012 Northern Italy earthquakes, the 2011 Tohoku Oki earthquake, Japan; and 2010/11 Canterbury-Christchurch earthquakes, New Zealand resulted in significant damage to built assets (Cubrinovski et al. 2014; Fioravante et al. 2013; van Ballegooy et al. 2014; Yasuda et al. 2013) and economic loss (Wood et al. 2016).

LIQUEFACT was a 42-month project funded by the EU to develop a better understanding of the earthquake soil liquefaction phenomenon across Europe and of the impact that an earthquake induced liquefaction disaster (EILD) event could have on the vulnerability and resilience of built assets and the communities they support. The LIQUEFACT project developed a range of technical (engineering, geotechnical, seismological) and business (resilience, economic, built asset management) tools to help technical and none technical stakeholders understand their susceptibility to an EILD event and evaluate the potential of mitigation interventions to improve their resilience through the LIQUEFACT Resilience Assessment and Improvement Framework (RAIF) (Jones et al. 2020). This paper reports a critical evaluation of the customisation process and an assessment of the potential of the UNDRR Disaster Resilience Scorecard for Cities to provide the basis for assessing the impact built asset mitigation interventions to an EILD event can have on community resilience. This paper presents the key technical and operational lessons learnt through the customisation process that can assist those applying the LIQUEFACT Scorecard or to those seeking to develop similar customised versions of the UNDRR Scorecard for use in different geographical settings or against different disaster scenarios. The LIQUEFACT Scorecard is one of four integrated tools developed by LIQUEFACT WP5 and WP6 to assess the impact of mitigation interventions at the individual built asset, portfolio of built assets and community level. As such, this paper should be read in conjunction with other papers presented in this special edition.

2 Literature review

2.1 Review of the factors affecting community resilience

The UNDRR definition of resilience “The ability of a system, community or society exposed to hazards to resist, absorb, accommodate, adapt to, transform and recover from the effects of a hazard in a timely and efficient manner, including through the preservation and restoration of its essential basic structures and functions through risk management.” (UNDRR 2017) was used by the LIQUEFACT project.

The concept of resilience was introduced by Holling (1973) to describe the ability of ecological systems to maintain their original state whilst absorbing changes and resisting external disturbance. Holling (ibid.) argued that fluctuations over time in a natural system could not be described by considering exclusively the behaviour of the system close to its equilibrium point (stability) but also needed to consider its ability to absorb changes generated by external stressors that, whilst they generate large fluctuations ultimately result in a more resilient system. In 1996 Holling explored further the behaviour of a system near its equilibrium point. He arguing that two aspects of stability need to be considered: one occurring close to the systems equilibrium point before the disturbance (engineering resilience); and the other moving the system to a new state of equilibrium (ecological resilience). In order to measure resilience, engineering resilience uses resistance to disturbance and speed of return to the equilibrium; whilst in ecological resilience the magnitude of the disturbance required to move the system to a new state of equilibrium is used. From a disaster risk reduction perspective both aspects of resilience influence our understanding of system behaviour, and hence community resilience to disaster events (Manyena 2006; Melkunaite 2016).

Whilst there are still arguments about the basic definition of resilience (Gallopín 2006; Matyas and Pelling 2012; Tiernan et al. 2019), there are fewer arguments about the characteristics of community resilience to disaster events with robustness, redundancy, resourcefulness and rapidity of systems expressed across technical, organisational, social and economic domains identified as critical components of resilient communities (Cutter et al. 2008; Cutter et al. 2010; Paton 2007; Paton et al. 2013; Paton and Johnston 2017; Tierney and Bruneau 2007). Cutter et al. (2010) applied the Disaster Resilience of Place model (Cutter et al. 2008) to a comparative assessment of community resilience between 736 counties and several metropolitan areas in the south-eastern United States of America, concluding that the DROP model provided a “broad brush” level of detail to allow the identification of overall patterns and intervention strategies that could improve the resilience of a region; and provide the basis to assess place-specific capabilities and interventions for enhancing disaster resilience. However, Cutter et al. (2010) also acknowledged the limitations of their model to accurately characterise location specific circumstances.

In 2019 Tiernan et al. (2019) published a review of themes in disaster resilience literature, effectively establishing a state-of-the-art of knowledge that described resilience as an overarching term that encompasses a range of system responses to disturbances, including the concepts of stability, recovery and adaptation. From the review Tiernan et al. (2019) identified three current significant research themes: social responsibility for resilience; risk management and the role of public-private partnerships as enabling mechanisms; and the emergence of adaptive resilience as a key concept to understand how community’s behave after a disaster event. Tiernan et al. (2019) also identified the inherent complexity in resilience theory and operational models that need to be addressed if resilience theory is to be translated into effective end-user (practitioner) tools. Indeed, the need to take a holistic view of community resilience, including the need to consider potential inter-relationships between factors/characteristics has been identified by many authors (Cutter et al. 2008; Cutter et al. 2010; Paton 2007; Paton et al. 2013; Paton and Johnston 2017; Tierney and Bruneau 2007) although very few have explored how such inter-relationships can be expressed and represented in practical applications.

The argument that community resilience to disaster events should be viewed as a complex system was explored in the CREW project (Hallett 2013) where the interaction between vulnerability, resilience and adaptive capacity of key community stakeholder groups were seen to either enhance or inhibit community resilience to a disaster event. As part of the CREW project, Ali and Jones (2013) explored the relationships between vulnerability, resilience and adaptive capacity and the complex socio-political context that frames community resilience and identified the need to improve awareness of the impacts of disaster events amongst those directly responsible for implementing mitigation decisions, as many stakeholder groups can exhibit risk-averse attitudes that delay decisions until more evidence is available, rather than investing in protective mitigation against future potential events. In this context, Ali and Jones (2013.) identified the crucial role that built environment professionals can play in making disaster science understandable and translating it into operative actions for the built environment.

Bruneau at al. (2003) also recognised the need to quantify infrastructure systems and community resilience as a complex process. Bruneau et al. (2003.) adopted an engineering perspective in which community resilience is measured as the difference between the ability of an infrastructure to provide services to a community prior to the occurrence of the event and the expected ability of that infrastructure to provide the services after the event. The factors they identified are mainly related to the physical infrastructures: the quality prior the event; the expected reduction in quality over time due to the occurrence of the event; the expected length of time that quality is below the pre-earthquake level; and the set of possible earthquakes and their probabilities of occurrence in the community.

Boon et al. (2012) also proposed the need to consider community resilience to disasters as a complex system applying Bronfenbrenner’s bioecological systems theory of resilience to organise the factors that enhance resilience into a hierarchy of systems (micro, meso, exo, macro and chrono) to assess both an individual’s resilience and estimate the overall resilience of a community to a disaster event. Boon et al.’s (2012) approach also provides a basis to measure the effects of mitigation interventions on individual and community resilience by considering their position within a spaciotemporal model.

Cavallo (2014) questioned whether a command-control approach (engineering resilience) could provide a complete picture of community resilience to disaster events, suggesting the need for community resilience to be viewed as a dynamic adaptive system (socio-ecological resilience). In such a system Cavallo (2014.) argued that recognising a series of critical thresholds exist and preparing for them was more important to community resilience than attempting to quantify the disaster event. According to Cavallo (2014) this action can enhance community resilience by implementing measures to increase the distance to the thresholds rather than focusing on speed of recovery. The arguments between using engineering resilience or socio-ecological resilience to understand community resilience were further explored in the IMPROVER project (Melkunaite 2016) who suggested that a hybrid view of resilience should be applied to model community resilience. This hybrid concept is consistent with the work of Holling (1973, 1996) and forms the theoretical basis of the LIQUEFACT project’s view of resilience where local context and system functionality, viewed from a spaciotemporal perspective that acknowledges interactions between system components (subsystems) informed the mitigation decision making process for evaluating the impact of EILD events on communities.

2.2 Modelling community resilience

Whilst there is general agreement on the factors/characteristics that affect community resilience, there is less agreement on how to integrate these into explanatory/predictive models and practical toolkits to assess community resilience, and to identify mitigation interventions to improve it (Sharifi 2016); this was one of the key objectives of the LIQUEFACT project. Two key issues arise. Can reliable, robust and consistent metrics be developed to measure each of the factors? What methods can be used to combine the factors into a reliable robust and consistent scorecard that reflects local circumstances and context?

In addressing these questions researchers (including: GOAL 2015; DPRAP/CoBRA 2013; Menoni et al. 2012; Bruneau et al. 2003; Kellett and Mitchell 2014; Resilience Alliance 2010; IFRC 2013; Ainuddin and Routray 2012) have developed a number of resilience toolkits that, whilst they differ in detail, follow the same generic approach. Most toolkits start by defining the system (domains/components/characteristic/factors) and hazard threat (event characteristics) being modelled. The system could be modelled as a simple linear system; as a hierarchy system; or as complex inter-connected system of systems. The hazard threat could be expressed at a single point in space and time or as a series of scenarios. Once the system has been contextualised, metrics (typically comprising nominal or ordinal scales) are developed that operationalise the domain, component, characteristic or factors and allow them to be scored against the hazard threat. Once scored the metrics are aggregated using appropriate (to the scales of measurement used to define the metrics) linear, statistical or quasi-statistical methods (e.g. summation; factors analysis; structural equation modelling etc.) and weighting factors (simple summation of factors or derived through a form of expert judgment that assess the relative importance of each factor to the overall resilience of the domain/component) are applied to provide an assessment of the resilience of the domain/component. The resilience scores for the domain/component are then aggregated (again using weighting factors) to score the overall resilience of the system to the hazard threat. Finally, the resilience of the system is compared to similar systems through a benchmarking process or is re-evaluated to take account of mitigation actions (individual metric scores are reconsidered to reflect the expected impact that a range of mitigation interventions would have on them).

Sharifi (2016) published a critical review of 36 specific community resilience assessment tools to identify the critical attributes that tools required to effectively measure community resilience at different physical scales (individual built assets through to national community levels) against different risk scenarios (single to multiple risks; natural to economic stressors). Sharifi (2016) categorised each tool against 15 characteristics to identify the critical characteristics for different applications. Sharifi (2016) identified six critical criteria for community resilience tools: accommodating multiple dimensions of resilience; accounting for cross-scale relationships; capturing temporal dynamism; addressing uncertainties; employing participatory approaches; and developing action plans. Although not considered in his analysis the Disaster Resilience Scorecard for Cities (UNDRR 2017) possesses all these criteria (to a greater or lesser extent) and this, taken together with the Scorecards coverage of the range of characteristics of community resilience identified by LIQUEFACT from literature (see Morga et al. 2020 for details) and by Cutter et al. (2010) made it the preferred choice for assessing community resilience to an EILD event. This choice was also supported by the fact that other projects are utilising the Disaster Resilience Scorecard for Cities in similar ways to assess community resilience to earthquakes (Burton et al. 2017).

2.3 The Sendai framework for disaster risk reduction

The Sendai Framework for Disaster Risk Reduction 2015–2030 (UNDRR 2015) advocates a coordinated, all-of-society approach to improve preparedness and response to disaster events; arguing that effective disaster risk reduction should be viewed as a shared responsibility between governmental, community, public and private sector stakeholders. To this end Sendai argues that the management of disaster risk: (1) should aim at protecting people and reducing losses to property (including cultural and environmental assets) and livelihoods: and (2) should be based on risk-informed decision making that integrates scientific information with local knowledge, including a multi-disciplinary understanding of how risks affect, and are interpreted by different stakeholder groups. To support the implementation of the Sendai Framework the UNISDR (2016) developed a scientific and technology road map to identify the expected science and technology outcomes needed to support the implementation of the Sendai Framework and provide the practical basis for the development of toolkits to assess disaster risk reduction. The Sendai Framework and UNISDR Science and Technology Road Map form the practical basis for the development of the customised UNDRR Disaster Resilience Scorecard for Cities developed by LIQUEFACT to reflect local conditions and circumstances that support a build-back-better philosophy to reduce future risk from EILD events (Morga et al. 2020). However, LIQUEFACT has also recognised the low attention given to engineering factors in the Sendai Framework (Booth 2018) which it addressed as part of the customisation and validation process.

2.4 The UNDRR disaster resilience scorecard for cities

The UNDRR Disaster Resilience Scorecard for Cities, which is based on Ten Essentials for Making Cities Resilient, developed to support the introduction of the Sendai Framework, is likely to be currently the most widely accepted community resilience toolkit. The Scorecard was developed to provide cities with a practical tool to identify their disaster resilience against each “Essential” and to identify areas that need mitigation interventions in order to improve their disaster resilience over time (UNISDR 2017).

The Scorecard (ibid.) consists of 118 disaster resilience evaluation criteria (subjects/issues) grouped into 10 Essentials (Table 1); which can be divided in the following macro groups: governance and financial issues (Essentials 1–3); planning and disaster preparation (Essentials 4–8); and disaster response and post-disaster recovery (Essentials 9–10) (Fig. 1). Each criterion is defined, and comments are provided as explanations. Criteria are scored against an indicative measurement scale (from 0 to 5, where 5 is the best practice) against a “most probable” and a “most severe” risk scenario that reflect the estimated hazard threat level in the assessed city/region. In order to score the criteria objective measures should be adopted but where these do not exist subjective assessments can be made. The aggregation of single scores can be done at the Essential Level to provide an indicative assessment of the resilience of each Essential to each scenario; and Essentials can be aggregated to provide an overall assessment of the city/region’s resilience to each scenario. Cities that have used the Scorecard have done so at three different levels: an initial scoping level; a detailed focused level (e.g. selected Essentials); or an in-depth assessment of all of a city’s resilience “Essentials” (Melner 2018).

Fig. 1
figure 1

(Source UNDRR 2017)

The UNDRR disaster resilience scorecard for cities.

Full size image
Table 1 Details of the subjects/issues addressed in the UNIDR Disaster Resilience Scorecard for Cities
Full size table

In a further review of the application of the Scorecard to over 200 cities and local governments (and 20 Model Cities—used to map the Ten Essentials against urban resilience critical success factors) Schofield and Twigg (2019) argued that for the Scorecard to be most effective it needs to be integrated in disaster reduction action plans in a way that supports the identification of mitigation activities to improve resilience and to be integrated into the wider hazard mitigation planning process, including integration into local business continuity and disaster risk management planning. This integration, they argued, includes the need to contextualise the Scorecard to reflect local circumstances (and risks) and to remove any criterion that do not apply. Schofield and Twigg (2019) also identified the need for those administering the Scorecard to undertaking preparatory research to better understand the risk that different hazards could have on each Essential criterion before presenting the Scorecard to workshop participants.

The LIQUEFACT project adopted the approach suggested by Schofield and Twigg (2019) to guide the development of a customised version of the Disaster Resilience Scorecard for Cities as the basis for assessing community resilience to EILD events and to inform the development of disaster risk reduction action plans through a Resilience Assessment and Improvement Framework (see Jones et al. 2020). The remainder of this paper describes the customisation process and validation activities of the EILD customised Scorecard.

3 Research methods

The research presented in this paper forms part of WP5 of the LIQUEFACT project. The aim of WP5 was to provide an integrating framework linking technical solutions (developed in WP2, WP3, WP4) with business and community solutions (developed in WP6 and WP7). In addressing this aim WP5 adopted a mixed-method research methodology, combining both quantitative and quantitative approaches to develop a holistic understanding of the impact that earthquake induced liquefaction mitigation interventions could have on built asset and community resilience. The research presented in this paper used a qualitative research methodology. A brief summary of the methodology, extracted from Morga et al. (2020), is given in this section. Full details of the methodology can be found in Morga et al. (2020).

A scoping literature review was undertaken to identify the factors affecting community resilience and the approaches used to model community resilience. Prior to the detailed customisation of the Disaster Resilience Scorecard for Cities a pilot study was undertaken to test the general principles behind a scorecard approach to assessing community resilience to EILD events. A short questionnaire survey was developed based on the generic factors identified in the scoping literature review to establish the degree to which different stakeholder groups could identify and score the relevance of a range of resilience factors. If the stakeholder groups could not distinguish between the factors then the scorecard approach would not provide the risk framing tool needed by the LIQUEFACT project to establish a communities antecedent resilience to an EILD event or provide the basis for evaluating the potential of ground mitigation interventions to improve community resilience. The pilot questionnaire was tested at the inaugural LIQUEFACT workshop held with representatives from different stakeholder groups (engineers, architects, geologists drawn from representatives of municipalities, local authorities, governmental institutions; academic institutions, and private consultants with recent knowledge of EILD events) from across the Emilia Romagna Region of Italy on October 3rd 2016. During the workshop 112 participants completed the short questionnaire where they scored the impact that they thought a range of factors (grouped into technical, organisational, social and economic domains) could have on community resilience to EILD events. Each factor was scored using a five-point Likert scale (“very low”, “low”, “medium”, “high” and “very high”). Participants were also asked to describe the direction of influence that they believed each factor had on community resilience using (i.e. “Positive”—as the factor increases/decreases, the resilience increases/decreases or “Negative“—as the factor increases/decreases, the resilience decreases/increases). The mean, standard deviation and modal scores and direction of influence by factor are shown in Table 2 (adapted from Morga et al. 2020). The UNDRR Disaster Resilience Scorecard for Cities was customised for EILD events through a series of workshops with external stakeholders and LIQUEFACT project partners (Morga et al. 2020). A discursive research methodology was used to gain a greater understanding of how different stakeholder groups perceived community resilience to EILD events; investigate the effectiveness and usefulness of the criteria presented in the UNDRR Scorecard to score resilience to EILD events; and to develop additional expert guidance to support the application of the Scorecard by non-technical stakeholders. Twelve group interviews (located in Italy or Turkey), comprising 45 individuals, were held with external stakeholders (practitioner engineers, surveyors, geologists, disaster managers, emergency responder and politicians) and LIQUEFACT project partners (academic/researcher engineers, construction technologists, geologists, and geophysicists) between May and December. 2018 Each group interview used the same structure. Following a general welcome, the interviewer confirmed that the purpose of the interview was to evaluate the suitability of the UNDRR Disaster Resilience Scorecard for Cities to assess community resilience to an EILD event; and where necessary, to provide modified statements that better reflected the impact that each of the criteria, addressed by the Scorecard, would have in the event of an EILD event. Each group interview was conducted under Chatham House Rules: “When a meeting, or part thereof, is held under the Chatham House Rule, participants are free to use the information received, but neither the identity nor the affiliation of the speaker(s), nor that of any other participant, may be revealed” (Chatham House 2020). Two interviewers facilitated the group interviewers: one for the interviews in Italy and the other for the interviews in Turkey. The interviews were conducted in the participants native language and each interviewer was a native speaker of the country in which the interviews were carried out. Both interviewers used the same standard protocol script and set of questions for the interviews. As these were focus group type interviews, the interviewer prioritised the narrative, providing further explanations or examples where interviewees struggled to understand or contextualise the criterion. All the interviews were audio recorded and then transcribed by the interviewers into English and summarised in an excel spreadsheet. Before each group interview all participants were sent the UNDDR Scorecard in order to give them the opportunity to familiarise themselves with the document. However, during the interviews the interviewer reviewed again the subject/issue; question/assessment area; indicative measurement scale; and comments sections for each criterion to ensure that all the group participants understood the reasons for the interviews. The group were then asked to consider and discuss each of the criterion in Essentials 1–4 and 6–10. Essential 5 (Safeguard Natural Buffers to Enhance Protective Functions Offered by Natural Ecosystems) was eliminated from the discussion following the first group interviews (with liquefaction experts) as the potential impact of liquefaction on the Natural Ecosystem was considered to be minimal in terms of its effects on community resilience. Essential 8 was also removed (except for criteria 8.1.1 and 8.1.2 as they focus on protective infrastructure and their maintenance) as LIQUEFACT researchers had identified (in a different aspect of WP5) the need to develop a much more detailed critical infrastructure scorecard which could reflect the localised impact that an EILD event would have on the performance of a range of critical infrastructure systems. For the remaining Essentials participants were asked to discuss the relevance of the criteria to community resilience to an EILD event. Where a criterion was considered relevant participants were then asked to provide comments to explain the reasoning behind their decision and, where necessary, suggest modifications to the Scorecard. Consensus was not reached in all the cases; however, all the views were recorded and analysed. After all those who wanted to express an opinion on the criterion had been given the opportunity to do so the interviewer moved to the next criterion and the process was repeated. Each group interview lasted approximately three hours. Analysis of the interview responses was carried out by three independent researchers and the results combined into a single version of the revised Scorecard. The relevance of each criterion was assessed by considering the relevance score given by each group to the criterion and the level of confidence that each group had in their ability to provide the score. Based on this approach each criterion was given a numeric score (mean of the relevance score across the 12 interview groups) which was converted into a qualitative ranking (high, medium-high, medium, medium-low, low) in the final version of the customised Scorecard. In addition to the relevance score for each criterion an analysis of the textual justifications provided by each group for its relevance ranking was performed to provide additional guidance to those applying the Scorecard in the future. The analysis of the textual data was done at both the criterion and domain levels. Whilst the criterion level analysis is included directly in the customised Scorecard (see Table 3 for an example taken from Essential 1—Organise for Resilience domain) the domain level analysis provides strategic guidance on how to apply the Essential to an EILD event scenario. (see Table 4 for an example from Essential 2—Identify, Understand and use Current and Future Risk Scenarios). Lastly, the customised Scorecard was tested and validated at a one-half day test and validation workshop conducted on 14th of October 2019 in Bologna, Italy. The aim of the workshop was to test and validate the refined set of criteria, items, indicators and metrics against a most-severe and most-probable EILD scenario, applied to the Emilia Romagna Region of Italy. During the test and validation workshop, stakeholders were asked to consider each of the items included within LIQUEFACT EILD Customised Disaster Resilience Scorecard for Cities and to rank the importance of each item using a 0 to 5 scale and to provide comments to explain their score. The workshop was attended by twenty-two stakeholders working for Regional authorities (Toscana, Liguria, Lombardia, Veneto, Emilia Romagna), District authorities, Municipalities and Civil protection, Geological, Seismic and Land Authority of Emilia Romagna Region and Agency for Reconstruction. The professions of the participants were: geologists; regional officials; department for Civil Protection official; urbanists; professionals; consultants; regional officials—seismic risk expert; cartographer; geologist for urban planification; environmental technical. Descriptive statistic was used to analyse the average scores and the differences in scores between the customisation interviews and the validation workshop (Morga et al. 2020).

The research received ethical approval from Anglia Ruskin University, Science and Technology Research Ethics Panel.

4 Results

4.1 Assessing community resilience to EILD events

The results of the pilot questionnaire (arithmetic mean, standard deviation and mode sores) suggest that whilst all the factors (except political leadership) were considered of some relevance to community resilience, those that addressed the ‘technical’ (3.70) domain were generally considered more relevant than those associated with ‘social’ (2.97), ‘economic’ (3.28) and ‘organisational’ (3.27) domains (missing data has been excluded from the analysis) (see Table 2). Further, whilst the ranking order of relevance was generally consistent across all respondent groups (all group’s ranked the technical domain as the most relevant), the Architect, Engineer and Geologist stakeholder groups rated all the factor domains as relatively more relevant than did the manager group (with the exception of Engineers that ranked the social domain lower than the Manager stakeholder group). This said, none of the stakeholder groups exhibited a wide standard deviation in relevance factor values which suggests that the generic range of community resilience factors identified from literature could form the basis of a toolkit to assess community resilience to EILD events. With the generic approach established the research team began the process of customising the UNDRR Disaster resilience Scorecard for Cities for use by regional stakeholders (Morga et al. 2020).

Table 2 Arithmetic mean, standard deviation, modal score and direction of relevance of a range of technical, organisational, social and economic factors to community resilience to an EILD event—all respondents (n = 112).
Full size table

4.2 Customising the UNDRR disaster resilience scorecard for cities for EILD events

The LIQUEFACT project used the UNDDR Disaster Resilience Scorecard for Cities as the basis for assessing community resilience to EILD events. The criteria; question/assessment area; indicative measurement scale; and comments sections outlined in the toolkit were developed by external stakeholders and LIQUEFACT project partners and validated by external stakeholders and the LIQUEFACT International Advisory Board members. A summary of the analysis, extracted from Morga et al. (2020), is given in this section. Full details of the analysis can be found in Morga et al. (2020).

Detailed analysis of the interview responses was carried out by three independent researchers and the results combined into a single version of the revised Scorecard. The relevance of each criterion was assessed by considering the relevance score given by each group to the criterion and the level of confidence that each group had in their ability to provide the score. If 50% or more of the respondents thought that the criterion was relevant, and they were confident in their judgement it was retained in the Scorecard and specific guidance on its application to EILD events was provided. If less than 50% of respondents thought that it was irrelevant, and they were confident in their judgement it was removed from the Scorecard. Criteria for all other combinations of relevance and confidence were considered to be of indeterminate standing, requiring further investigation to reflect local context and circumstance. The analysis showed that all the criteria for Essential 1 (organise for resilience) and all but one criterion (update process—which was considered relevant but where there was disagreement with the 5 year update timescale—where the consensus was to update as better liquefaction macrozonation/microzonation data becomes available) for Essential 2 (identify, understand and use current and future risk scenarios) were considered relevant when evaluating community resilience to an EILD event. Essentials 3 (strengthen financial capacity for resilience), 7 (understand and strengthen societal capacity for resilience), 9 (ensure effective disaster response) and 10 (expedite recovery and build back better) were all indeterminate, requiring further investigation of local context and circumstances to establish the specific impact that an EILD event would have on community resilience. The general consensus amongst interviewees was that no additional consideration would be needed for EILD events providing that liquefaction impacts were considered as part of a general earthquake disaster scenario. Essential 4 (pursue resilient urban development) was the only Essential that showed a bipolar response; respondents either thought that the criteria were definitely relevant (all criteria except 4.1.1—potential population displacement, and criterion 4.3.3—sustainable building design standards) or definitely irrelevant to evaluating the impact that an EILD event would have on community resilience. The primary reasons given for those criteria identified as irrelevant were the very localised nature of EILD events and the lack of a direct link between green building standards and reduced vulnerability to EILD impacts. Most of the criteria for Essential 6 (strengthen institutional capacity for resilience) were considered irrelevant except for criterion 6.1.2 (private sector links), criterion 6.1.3 (engagements of the insurance sector) and criterion 6.1.4 (civil society links) which were all indeterminate. Again, the localised nature of EILD impacts was the most commonly cited reason for excluding criteria (note: some respondents did say that liquefaction should be included as part of general earthquake awareness education and training programs). Two criteria (adequacy of protective infrastructure and effectiveness of maintenance) of Essential 8 (increase infrastructure resilience) were explored in detail in the interviews and both were considered relevant to improving community resilience to an EILD event. An early decision by all LIQUEFACT researchers (following the first group interview—which was conducted as a pilot study to prove the research approach) had identified the limitations of scoring all of the other criteria of Essential 8. From one side, they were perceived to be to high-level to reflect the localised impact that an EILD event would have on the performance of a range of critical infrastructure systems, but on the other side it was noted that only stakeholders with specific knowledge about the specific critical infrastructure could answer them. Therefore a decision was taken to develop a separate site-specific critical infrastructure resilience assessment tool. This also in part addresses the concerns raised by Booth (2018) about the lack of engineering factors in the Scorecard. Finally, the relevance score for each criterion was converted into a numeric score (% of ‘Yes’ scores to total scores excluding missing values) to reflect the relevance level score across the 12 interview groups. This score was then converted using a simple scaling factor (assuming an equal distance between the qualitative categories) into a qualitative ranking (high, medium-high, medium, medium-low, low) and included as guidance in the final version of the customised Scorecard for those performing a similar customisation process to reflect their local context and circumstances.

In addition to the relevance score for each criterion an analysis of the textual justifications provided by each group for its relevance ranking was performed to provide additional guidance to those applying the Scorecard in the future. The comments for each criterion were then analysed independently by three researchers (one of whom designed and managed the interview process and two who were independent of the process) to identify common themes between the interview groups and crosscutting themes between the criterion at the domain and sub-domain level (note: topline Essentials represented a domain which is comprised of sub-domains of criterion). An example of the domain level guidance is shown in Table 3. An example of the criterion level analysis for Essential 1 is shown in Table 4.

Table 3 Example of the domain level guidance for the application of the customised EILD event Scorecard.
Full size table
Table 4 Summary of the criterion level guidance and indicative relevance score included in the customised EILD event Scorecard.
Full size table

A more detailed analysis of the workshop interviews, along with the full version of the customised scorecard can be found in Morga et al. (2020).

4.3 Validation of LIQUEFACT EILD customised disaster resilience scorecard for cities

The scores of each Essential obtained during the interviews with LIQUEFACT researchers and external stakeholders and the validation workshop were compared, and results are reported in Table 5.

Whilst the absolute relevance score varied between the development phase interviews and validation phase workshop the relevance ranking of Essentials was generally consistent between the two phases. Both the development phase interviews and validation phase workshop ranked Essential 1 (organising for resilience) and Essential 2 (identify, understand and use current and future risk scenarios) as those most relevant when considering the impact of an EILD on community resilience. Essential 9 (ensure effective disaster response) and Essential 10 (expedite recovery and build back better) were ranked as the least relevant when considering the impact of an EILD on community resilience. The only significant areas of divergence between the development phase interviews and validation phase workshop were in the relevance of Essential 3 (strengthen financial capacity for resilience) which was ranked equal third relevant by the validation phase workshop but sixth relevant by the development phase interviews and Essential 8 (increase infrastructure resilience) which was ranked second equal relevant by the development phase interviews and fifth relevant by the validation phase workshop. All the other Essentials, with the exception of it Essential 8 (increase infrastructure resilience—where only two criteria were considered) were considered of mid relevance by both the development phase interviews and validation phase workshop.

Table 5 Comparison of absolute relevance course and relevance rankings for the customised EILD scorecard between the development and validation workshops
Full size table

Where there were large differences in absolute relevance scores between the development phase interviews and validation phase workshop, or where the validation phase workshop rated the relevance of an Essential higher than the development phase interviews, researchers examined the textual comments given by both sets of respondents to justify their scores. For Essential 1 the differences in absolute score were primarily the result of differences in the scoring the relevance of the ‘Organisation, Participation and Coordination’ criterion. For Essential 2, the main difference was in the scoring of the relevance of ‘Updating of Scenario, Risk, Vulnerability and Exposure’. For Essentials 4, 6, 9 and 10 all the criteria were considered less relevant by the validation phase workshop when compared to the development phase interviews. No clear trend could be observed between the development phase interviews and validation phase workshop for the difference in relevance scores for Essential 8. Due to time constraints Essential 7 could not be fully considered during the validation phase workshop.

An extract from the final customised version of the UNDRR Disaster Resilience Scorecard for Cities customised to reflect and EILD event is shown in Table 6 (extract of a criterion taken from Essential 1). The full customised scorecard can be found in LIQUEFACT Deliverable 5.4 (Morga et al. 2020).

5 Discussion

The aim of this paper was to report the technical and operational lessons learnt in customising the UNDRR Disaster Resilience Scorecard for Cities for EILD events.

A detailed review of literature identified the factors affecting community resilience and approaches to modelling built asset and community resilience. The findings from the literature review formed the basis of a questionnaire survey of stakeholder groups familiar with the impact that EILD events which confirmed the relevance of the factors; the ability of stakeholder participants to distinguish between resilience factors; and the appropriateness of a scorecard methodology to provide the risk framing tool needed by the LIQUEFACT project. Given its widespread use, the UNDRR Disaster Resilience Scorecard for Cities (Melner 2018; Scofield and Twigg 2019) was chosen as the basis of the LIQUEFACT scorecard. Group interviews with external stakeholders and LIQUEFACT project partners considered the relevance of each of the 118 criteria in the UNDRR Disaster Resilience Scorecard for EILD events, providing a relevance score, and detailed guidance on how to interpret these criteria in the context of an EILD event. The customised version of the UNDRR Disaster Resilience Scorecard for Cities for EILD events was validated in a workshop with a group of stakeholders who had not been involved in its original development. Whilst the validation process confirmed the general applicability of the customised scorecard to assess community resilience to an EILD event, it also highlighted the difference in interpretation of the relevance of each criterion between the participants of the group interviews used to develop the customised scorecard and those in the validation workshop. As such those intending to apply the EILD customised version of the UNDRR Disaster Resilience of Cities Scorecard will need to contextualise it to reflect their city specific location and context before using it to assess their resilience to an EILD events.

The UNDRR Disaster Resilience Scorecard for Cities has been applied in over 2,880 cities (Melner 2018) and whilst its success in providing a practical framing for discussing disaster risk reduction issues is acknowledged, the need to contextualise the scorecard to reflect local hazard characteristics and circumstances is also identified (Schofield and Twigg 2019). The UNDRR Scorecard represents a high level, top down, assessment methodology of the impact that a disaster event can have on 10 Essentials, grouped by their impact on corporate/city governance, integrated planning, and response planning procedures. Each Essential comprises a range of criterion that are scored on a 0–5 qualitative scale to reflect their baseline resilience and identify and prioritise actions and projects to improve resilience. As such, the UNDRR Scorecard should provide the basis for evaluating the baseline resilience of city/regions to an EILD event. However, there is one significant difference between EILD events and other types of disaster events that have been successfully modelled using the UNDRR Scorecard (which tend to be widescale events that have an impact across a wide geographical area); namely the very localised nature of EILD event impacts which tend to be built asset or site specific. The results of the customisation process suggests that whilst the UNDRR Scorecard can be effectively customised to support communities/municipalities to understand the impact of built asset/site specific EILD mitigations, there are a number of technical and practical issues that have to be addressed, including the need to consider local context (different disaster management structures, governance models) and circumstances (past experiences and attitudes to risk).

Participants in both the development phase interviews and validation workshop found it difficult to assign EILD attributes to the general assessment criteria. This said, it was clear that those Essentials that deal with corporate and city governance (1, 2 and 3) are considered fundamental to a city’s resilience planning for an EILD event. There was widespread agreement amongst participants of the importance of applying macro-zonation/micro-zonation maps and studies that explicitly address EILD risks to critical assets and to use fragility curves to assess the impact that and EILD could have on their performance following an EILD event. There was also general agreement on the need to address the financial costs of improving critical assets resilience and ensuring that costs are budgeted for as part of disaster management and business continuity planning. The Essentials that support integrated planning (4, 6, 7) are less influenced by the specifics of an EILD event and as such, providing that EILD events are integrated into general earthquakes disaster scenarios (e.g. form part of a most probable and most severe earthquake scenarios) both the development phase interviews, and validation workshop participants thought that little additional criteria would be required to customise the scorecard for an EILD event. Again, the need was recognised to implement macro-zonation maps, micro-zonation analyses, and site-specific investigations within planning guidelines and building codes risks assessments. Essential 8, which also form part of integrated planning, was considered to be critical to community resilience to an EILD event and, was considered to not be sufficiently addressed in the UNDRR Scorecard. As such, the LIQUEFACT project developed its own critical infrastructure scorecard (LIQUEFACT D5.4 2020) which can be integrated into the EILD customised UNDDR Scorecard to provide an overall assessment of a city/region’s resilience to an EILD event. The Essentials that support response planning were also considered to be less influenced by the specifics of an EILD event and providing that liquefaction was considered a part of a general earthquake scenario both the development phase interviews, and validation workshop participants thought that no additional information was required for an EILD event.

The customisation process provides a number of lessons for anyone undertaking a similar process, not least of which is the time and effort required to achieve the customisation process. Many interviewees commented on the complexity of the UNDRR Scorecard and on the time commitment, cost and logistics that would be required by cities to undertake the assessment for EILD events. This concern is consistent with that identified by Schofield and Twigg (2019) and again reinforces the need to undertake significant pre-stakeholder workshop preparation to not only contextualised the scorecard for local hazard and context but also prepare examples (through the use of secondary data) to show how each criterion could be applied to local circumstances. Such preparatory work would probably be best undertaken by a small group of people who have experience in both the specific hazard scenario being considered and the city being assessed. To this end the authors suggest the following practical approach to customising the UNDRR Scorecard.

  1. 1.

    Undertake detailed preparation work to fully understand the disaster scenario from both an engineering and community perspective as recognised by Booth (2018). A great deal of time will be spent in the stakeholder workshops arguing about the meaning of the UNDRR criteria and their specific impact on the city/region. The workshop facilitator needs to able to lead these discussions in an informed manner. Many workshop participants are likely to find the language and terminology used unclear or presented in a way that they find difficult to contextualise (Scofield and Twigg 2019). In the LIQUEFACT project this manifested itself through stakeholder bias and subjectivity, where each stakeholder group believed that their interpretation of the impact that an EILD event would have on each criterion was the correct one. It took firm leadership from the facilitator to ensure that the loudest voice (stakeholder group) in the room did not dominate the scoring.

  2. 2.

    During the stakeholder workshop removing criteria from the Scorecard if the consensus opinion is that they are not relevant can be beneficial. This will not only save time during the application phase but will also increase stakeholder confidence in the customisation process. Whilst respondents had some concerns about the applicability of some criteria to address specific liquefaction issues, and about the apparent double counting of some issues between Essentials, they did agree that a customised version of the UNDRR scorecard could be used to assess community resilience to an EILD event.

  3. 3.

    In conjunction with 2) the reasons for removing a criterion, along with any consequential action can be several (e.g. need to ensure that this issue is covered elsewhere in the Scorecard or forms part of a wider disaster scenario). In the case of the LIQUEFACT Scorecard this invariably meant ensuring that liquefaction was considered as part of a general earthquake scenario or liquefaction-specific mitigations actions were included as part of general earthquake mitigation plans.

  4. 4.

    Do not be concerned if at the start of the workshop time is spent by stakeholders questioning each other about the potential impacts of the disaster scenario. One of the strengths of the customised UNDRR scorecard approach is that it raises awareness of the impacts of the disaster scenario but also identifies inter-relations that would be missed if only one stakeholder group were involved. What you do need to be aware of is bias. In all the LIQUEFACT workshops there was clear evidence that each stakeholder group thought that their interpretation of the impacts was the most likely to be correct.

  5. 5.

    Ensure that the workshop also identifies what additional mitigation interventions (technical, operational, economic, social, governance, etc.) are needed to improve the resilience score of individual criteria or the overarching Essential. In some cases additional tools may need to be used to draw a complete picture of a city/region resilience. In LIQUEFACT this was achieved through a separate scorecard for critical infrastructure resilience and close integration of the LIQUEFACT Scorecard with a resilience assessment and improvement framework (Morga et al. 2020). Together these provide the evidence to support the business case for mitigation actions.

In addition to the difficulty that workshop participants had interpreting the relevance of the UNDRR scorecard criteria in the context of an EILD event, they also found it difficult to differentiate between the impact on community resilience of ground shaking and that of ground liquefaction. It had originally been assumed by the LIQUEFACT project that it would be possible to undertake a localised study of the impact that earthquake induced liquefaction would have on community resilience without the need to consider the wider impacts caused by ground shaking. The results of both the development interviews and validation workshop suggest that this is not the case. With this in mind the authors recommend that the customised scorecard presented in this paper is applied as part of a wider assessment of the impact that an earthquake scenario would have on community resilience. If the UNDRR Disaster Resilience for Cities Scorecard has already been applied to a city to assess its resilience to earthquakes, then the EILD customised Scorecard can be retrospectively applied and the results combined with those obtained from the earthquake scenario analysis. If the city has yet to undertake an assessment of its resilience to earthquakes, then the customised EILD Scorecard can be applied alongside the generic UNDRR scorecard. In both these cases the generic UNDRR Scorecard and the EILD customised Scorecard would need to be jointly customised to reflect a city’s local context and circumstances.

The LIQUEFACT customised UNDRR Scorecard forms part of the wider liquefaction resilience assessment and improvement framework developed to provide an overarching theory to integrate the scientific and engineering work of the project with the business models and tools needed to evaluate the potential benefit of earthquake induced liquefaction disaster mitigation measures to improve community built asset and community resilience (Jones et al. 2020). Both the LIQUEFACT Scorecard and the RAIF view community resilience to an EILD event as a complex adaptative system where risk reduction is shared between multiple stakeholders that need to act together to reduce the impact of an EILD event on both built assets and the wider community (Tiernan et al. 2019). To this end the LIQUEFACT customised Scorecard (and the RIAF) have used Cutter’s Disaster resilience of Place model (Cutter et al. 2008) to link the impacts of a localised disaster event (earthquake induced ground liquefaction) to the wider community. The LIQUEFACT customised Scorecard addresses multiple dimensions of resilience, applied across scales (from individual assets through to a portfolio of assets to the community level) to establish baseline resilience to an EILD event and identify the potential improvement in resilience to future EILD events as a consequence of different mitigation intervention options and different hazard scenarios. Further, the customisation process requires the active participation of multiple stakeholders to interpret the Scorecard for their city/region develop and evaluate specific mitigation intervention action plans to improve future resilience. As such the LIQUEFACT EILD Scorecard possesses the six critical criteria for community resilience tools identified by Sharifi (2016). This process the authors believe justifies their original assumption that the UNDRR Scorecard can be applied for a bottom-up assessment of community resilience where local impacts on built assets can be assessed and interpreted against a wider community resilience agenda.

Whilst this paper has presented the background to the EILD Scorecard, its full application forms part of a wider study that integrates the EILD Scorecard into a Resilience Assessment and Improvement Framework (Jones et al. 2020) and a Built Asset Management planning tool that integrates mitigation to EILD events as part of the building rehabilitation process. The development of the Built Asset Management tool is work in progress which will be reported in the future.

Table 6 Extract of a typical criterion from the LIQUEFACT customised Scorecard showing the additional guidance and scoring criteria.
Full size table

6 Conclusions

Modelling community resilience to EILD events is a complex process that involves understanding the impact of liquefaction on the determinants/attributes that are known to affect community resilience to disaster events. In particular there is a need for greater understanding of the disaster risks (hazard characteristics, exposure, and impacts) through hazard specific assessment tools such as macro-zonation maps, micro-zonation analyses, and site-specific investigations and of the procedures in place (plans, competencies, guidance and coordination) to mitigate the impact of these risks; both before, during and after (recovery and reconstruction phases) a disaster event.

The LIQUEFACT project developed an EILD customised version of the UNDRR Disaster Resilience Scorecard for Cities for use alongside a resilience assessment and improvement framework to assess the impacts that EILD events on built assets and community resilience. Whilst the customisation process confirmed that the UNDRR scorecard could provide the basis for a bottom-up assessment of built asset and community resilience, it did identify a number of issues that those seeking to apply the customised scorecard need to consider:

  • There was a clear need to customise the UNDRR Scorecard before it could be applied to an EILD event. The customisation process not only removed several criteria from the assessment but also provided the opportunity to explore the links between earthquake induced liquefaction and community resilience. This said, the customised UNDRR Scorecard will need to be contextualised to reflect local context (e.g. different disaster management structures, governance models) and reflect local circumstances (e.g. past experiences and attitudes to risk). In contextualising the customised scorecard consideration also needs to be given to the scale of the town/city/region being assessed. Due to the time and cost of contextualising the scorecard the authors suggest that it is only really applicable at the city/region scale.

  • The customisation process is not straightforward, and the authors propose a four step process for managing stakeholder engagement with the customisation process.

  • Interpreting the impact of hazards on the attributes/determinants of community resilience is difficult, even for experienced stakeholders. A significant amount of preparation is required before stakeholder workshops to facilitate meaningful discussions between stakeholders to ensure consistency in scoring criterion. Even with significant preparation facilitators should be aware of the potential bias and subjectivity that different stakeholders apply to their scoring.

  • The process of customisation significantly raised awareness of the links between an EILD event and community resilience. Whilst each stakeholder group was familiar (to a greater or lesser extent) with the implications of liquefaction to their system they were generally not aware of the implications that an EILD event would have across stakeholder groups systems.

  • The problems of local context and circumstances make direct comparisons between cities/regions very difficult. This said, the customised EILD scorecard can allow individual city/region mitigation interventions to be identified and evaluated when it is used in conjunction with the LIQUEFACT resilience and assessment and improvement framework (detailed in an accompanying paper in this special issue). As such the LIQUEFACT customised Scorecard does provide a useful tool for those assessing community resilience to EILD events.

The LIQUEFACT Scorecard is intended to supplement the UNDRR Disaster Resilience Scorecard for Cities, and not replace it. The LIQUEFACT Scorecard is intended to provide guidance to those facilitating the application of the UNDRR Scorecard to earthquake scenarios. To this end, the LIQUEFACT Scorecard should be used to provide guidance to stakeholders (or those facilitating stakeholder discussions) on how to consider the specific impacts of EILD events alongside ground shaking. Whilst the LIQUEFACT Scorecard provides detailed guidance at the criterion level (on what needs to be considered from an EILD event perspective) and on the relevance of the criteria to improving community resilience to an EILD event; these must be considered indicative, requiring interpretation at the local level to reflect local context (e.g. attitudes to risk etc.) and circumstances (e.g. past experiences etc.). This interpretation at the local level should take place following consultations with the different stakeholder groups involved in the assessment of the different Essentials. The supplemented UNDRR scorecard should then be applied in line with the guidance provided by UNDRR (available at: https://www.unisdr.org/campaign/resilientcities/assets/toolkit/Scorecard/UNDRR_Disaster%20resilience%20%20scorecard%20for%20cities_Detailed_English.pdf ).

Finally, although this paper has focused on customising the UNDRR disaster resilience scorecard for cities to reflect an EILD event, the authors believe that the approach they have explained can also be applied to those who seek to customise the UNDRR Disaster Resilience Scorecard for Cities to reflect other disaster scenarios.