Indicator-based resilience assessment for critical infrastructures – A review

power systems, play an indispensable role in our society due to their importance for maintaining critical societal functions, economic organisation and national defence. Therefore, the popularity of “ critical infrastructure resilience ” has exploded in both academic and policy discourses. Indicator-based assessment is a convenient and common tool to help understand, analyse and improve CI resilience in the scientific field. This paper produces a state-of-the-art review of the existing indicator-based assessment of CI resilience. After a terminology presentation, which helps clarify the objective of this study, this paper will show: 1) two methods for selecting the current scientific papers applying indicators to assess CI resilience; and 2) analysis of the indicators in these papers based on the study objective. The results show that there are many indicators and they do not have a uniform standard system, which means an indicator system for CI resilience assessment must be established.


Introduction
On 28 February 2022, the Intergovernmental Panel on Climate Change (IPCC) of the United Nations published a new report, Climate Change 2022: Impacts, Adaptation and Vulnerability Pörtner et al. 2022, aiming to assess the impacts of climate change, by looking at ecosystems, biodiversity, and human communities at global and regional levels.This report sets forth that most global territories are facing increasing adverse impacts of climate change on infrastructures and suggests integrated action for climate resilience to avoid natural disasters, including planning and investment in urban infrastructure.Meanwhile, the planet is also confronted with a growing number and magnitude of technological and technical disasters worldwide.For example, on 4 August 2020, an explosion in the main port in Beirut because of poor management of stored products, resulted in damage to many local infrastructures.In particular, three of Beirut's hospitals were wrecked by the blast Dyer, 2020, and two others were damaged, which added to the performance disruption of hospital infrastructures already facing the COVID-19 crisis.Infrastructures, especially critical infrastructures (CIs), including buildings, roads, schools, and hospitals, play a central role in modern city management when facing both technological and natural disasters.Therefore, in recent years, the popularity of "CI resilience" has exploded in both academic and policy discourses.The implemented actions for improving CIs based on

Methodology and structure
The method used to develop this review consists of two principle steps that build two sections (sections 3 and 4) of this article, with which the reader could find the specific methods concern.In considering the aims of this study, these two steps are used respectively to define the significant elements to be investigated of this study, and to identify suitable scientific papers to be analysed (see Fig. 1).Both two steps are based on current extensive literature available relating to indicatorsbased assessment for CIs resilience.
As indicators-based resilience assessment is a broadly used method in science and practice, a large number of factors or elements in the current papers are worth noting.To define the valuable elements that this study should investigate in each reviewed paper, the step in section 3 is to give a brief overview of the key sectors involved in reference to highly recognised scientific papers.The selection of these highly recognised papers was based on the knowledge and interview of a dozen authors of this work, all from the relevant sectors.Section 4 shows the process of paper searches.Two methods are presented for identifying useful scientific papers: (1) the common search method through keywords on electronic scientific websites; and (2) the reference search method based on some high-level review papers, thereby addressing some of the weaknesses of Method (1).Section 5 presents the results in tables and graphs.In section 6, the study further analyses the results based on a discussion of the elements identified in section 3.

Key sectors and focused elements
The study of CIs resilience has developed rapidly in the last decades and indicator-based assessment has become an important part of it.The scientific team for this study considers the work to involve three sectors: "Critical Infrastructures", "Resilience", and "indicator-based assessment".Moreover, the group has identified numerous highly recognised scientific papers in these three sectors after several exchanges.This section, taking into account these highly recognised papers, gives brief overviews of the common or significant aspects of these three sectors.As a result, based on this information, eight elements were elected to be investigated.

Critical infrastructures
There is no universally accepted standard for the classification of CIs.One of the reasons for this is that internationally there is no commonly accepted definition of CIs.Nevertheless, all definitions emphasise the contributing role of infrastructure to society or the potentially debilitating effect in the case of disruption CIPedia, 2020.For Brown et al. (2006), infrastructures that represent a significant public investment and where even minor disruptions can degrade the performance of systems and cause significant societal damage can be called critical infrastructures.Barroca et al. (2012) consider an infrastructure (i.e., a set of facilities providing services necessary for a city to function), as critical if its malfunctioning threatens the security, economy, lifestyle or public health of a city, region or even a state.For U.S Department of Homeland and Security U.S Department of Homeland and Security (2022), the infrastructure system, "whose incapacity or destruction would have a debilitation impact on the defence and economic security", could be considered critical.The European Commission identifies CIs as those physical and information technology facilities, networks, services and assets that, if disrupted or destroyed, would have a serious impact on the health, safety, security or economic well-being of citizens or the effective functioning of governments in European Union countries EUR-Lex, 2010.The disruption of infrastructure would have a serious impact on the well-being, health, and safety of citizens and weaken the whole defence and economic organisation Rinaldi et al. 2001;Serre and Heinzlef, 2018).
According to a review article by Moteff et al. (2003), it is difficult to This study suggests using 21 sectors defined by Proag (2014).Because, according to his study, resilience is considered important in these CI sectors (see Table 1).

Resilience and CIs resilience
Similarly, the term "resilience" is multidisciplinary and difficult to define because of its very broad use.Etymologically, resilience comes from the Latin, resilio, resilire, which means a return and the ability to resume.Many resilience studies argue that Holling (1973) first introduced the concept for studying ecological science and defined "resilience" as a persistent ability to absorb change and disturbance and still maintain the same state variables.Over time, a series of interpretations of resilience has been presented.As the definition of the term resilience is not the focus of this article, some relevant review studies Manyena, 2006;Meerow et al., 2016;Hosseini et al., 2016;Martin-Breen and Anderies, 2011;Curt et al., 2018;Heinzlef et al. 2022 are suggested here without further discussion.Overall, resilience has little orthodoxy in its conceptualization and application and becomes problematic when trying to measure it Cutter, 2016.In the field of CIs, resilience has no broadly accepted definition either, but is highly related to engineering or socio-technical science Cutter et al., 2010;Smith and Stirling, 2010;Mottahedi et al., 2021).For infrastructures exposed to disasters and interdependent in modern society, the definition of resilience is extended with multiple aspects and perspectives.Among the various interpretations of resilience, this study favours the one that accords with the highly accepted perspective in the scientific community, focusing on engineering resilience and sociotechnical dimension.

Properties and capabilities of CIs
The value or level of CI resilience can be described through the properties (like safety, sustainability, etc.) or capabilities of CIs from the narrower interpretation.Capabilities of a system could be capacities, characteristics, abilities, resources and knowledge Makadok, 2001;Renn, 2008;Hills, 2005;Eriksson and Juhl, 2012;Lindbom et al., 2015).Both property and capability are indeed used frequently for CI resilience.Leveson et al. (2017) focus on the safety of the socio-technical system for resilience assessment.Bruneau et al., (2003) emphasise the importance of five properties (capacity, robustness, redundancy, resourcefulness and rapidity) in engineered systems.Hollnagel (2014) mentions responding, monitoring, learning, anticipating ability, sustainability and safety properties, and required knowledge for the resilience of the built environment, including infrastructures.Barroca and Serre Barroca and Serre, (2013)believe that cognitive competence (i.e.having enough knowledge of CI and potential hazards) is one of the most essential aspects for infrastructure resilience.Vulnerability and fragility, highly related to resilience, are two characteristics of CIs that describe the damaging effects of a hazard Proag, 2014;Argyroudis et al., (2019).Furthermore, from an organisational point of view, for CIs exposed to natural hazards, resilience contains characteristics related to mitigation, planning and prior disaster experience Cutter et al., 2010. Ouyang et al. Ouyang et al., (2012-a.)highlightthree capacities for urban infrastructures: resistant, absorptive and restorative.Concerning recovery ability, the availability of resources in the CIS system has been usually mentioned (Turniquist and Vugrin, 2013;Ouyang and Wang, 2015).

Temporal stages of resilience scenario
Resilience could also be described through the change in performance or function over time.Therefore the resilience scenario presents with different temporal stages.Based on the existing theories of CI resilience (Ouyang et al., 2012-a. Ouyang et al., 2012-a.;Francis and Bekera, 2014;Tendall et al., 2015;CEREMA, 2020;Yang et al., 2022-a., this study divides the resilience scenario into four stages (see Fig. 2): • pre-event stage (PrES): from the occurrence of a hazard to the beginning degradation of the function of CIs, • during event stage (DES): from the beginning of the degradation to the maximum degradation of the function of CIs, • post-event stage (PoES): from the maximum degradation of the function of CIs to the function returning to the level of the pre-event stage or recovering to an ideal state, structure or property (but still lower than the original state), and • next event preparation stage (NEPS): from the function returning to the level of the pre-event stage to the occurrence of the next shock on CIs.This stage emphasises the ability of learning and improving from experience.
The NEPS in the former scenario is highly related to the PrES in the next scenario.In addition, it is necessary to emphasise that the ability of improvement may affect different following stages and scenarios.In each scenario, performance should improve at least in one stage compared to previous scenarios, thanks to the ability to learn and improve from experience.

Effects of implemented actions
As Barroca and Serre (2013) argue, a resilient CI should have different aspects of capabilities and involve actions to improve its capabilities.Potential action analysis helps identify decisions that should be taken to reconcile objectives and constraints in the best possible manner Alderson et al. (2015).Many researchers emphasise the benefit of implemented actions in a single scenario.Zhang et al. (2015), Zhang and Wang (2016), Ruas et al. (2018) recognise, for engineering-based systems, the effects of preparedness action at PrES and adaptive response actions at DES to recover efficiently and effectively.Zhang et al. (2015), Sharifi and Yamagata (2016) mention the cost-benefit of actions in discussing action efficiency.Madni and Jackson (2009) consider resilience actions as immediate or short-term actions while adaptation implies long-term learning.The latter highlights that the benefit of implemented action beyond a single scenario corresponds exactly to the improved capacity described above.Meanwhile, due to the interaction of CIs and cities' components, the actions of one CI potentially bring unexpected side effects on itself Leveson et al., 2017;Costella et al., 2009;Hollnagel and Woods;2017), even on the connected urban components Robert and Hémond, 2012;Serre and Heinzlef, 2018).From the point of view of organisational management, the positive or negative impact of an action is related to issues of efficiency and safety, which determine the degree of satisfaction of the action Atkinson et al., 2006;Tillement et al., 2009;Costella et al., 2009).Therefore, the effects of implementing actions should also be a key aspect during CI resilience assessment.

Interdependence
Recently, more and more CI resilience studies discuss the connections and interdependencies, which are primarily between CIs Rinaldi et al., 2001;Reed et al., 2009;Ouyang and Wang, 2015;Serre, 2016), sometime between CIs and non-technical components in human-environment Bambara et al., 2015;Yang et al., (2022-b).. Human-environment include physical, atmospheric, biological, social, economic and politic components, conditions, and factors which impact the state, condition, and quality of living conditions, employment, and health Academies and of Sciences, Engineering, and Medicine., (1992) .Nan and Sansavini (2017) assess resilience by taking into account infrastructure interdependence, which can correspond to one infrastructure (internal interdependency) or more infrastructures (external interdependency).Going beyond CI sectors, Markolf et al., 2018 rethink the resilience of the interactions between infrastructures and socioecological components.Similarly, Yang et al. (2022-a) believe that a resilient infrastructure should have the ability to manage multiple equilibriums with other urban systems.In modern society, due to the interdependencies of all built environments, the damage of one single CI, like physical disruption or dysfunction, frequently affects other urban components.The resilience of interdependence between infrastructures requires each system to guarantee its functionality when faced with the failure of other systems Serre and Heinzlef, 2018.This trend of CI resilience comes from the awareness of the cascading effects due to interconnected urban components.Increasing hazards require the urban system to cope with potential cascading effects after negative consequences on CIs.Furthermore, from a consequence-based approach, the negative effects caused by used resources should also be taken into account in the analysis of interdependencies Robert and Hémond, 2012.This opinion is equally related to the side effects of implementing the aforementioned actions.
Overall, CI resilience could be described through different properties, capabilities, stages and potential effects (positive, negative, side or cascading) in considering implementing actions and interdependency.The potential that occur after a disaster affects an infrastructure may occur in the same infrastructure or its connected urban components.The scope of urban components is broad, and could be categorised into socioeconomic systems, government organisation, nature and energy flow systems, and technical-infrastructural systems Yang et al., 2022-b..According to some historical events, the origin of catastrophic impacts of CIs (disruption, construction, action, etc.) have already occurred on socio-economic systems Chang, 2016;Tatano and Tsuchiya, 2008;Rose and Wei, 2013), government organisation Marples, 2006;Rania et al., 2019;Duffaut, 2013), nature and energy flow systems Gyau-Boakye, 2001;Buesseler et al. 2017;Aldeberky, 2007).

Indicator-Based assessment
Resilience assessment is a popular and common method in resilience studies due to its effort to evaluate and characterise the characteristics of a complex system Tang, 2019.Assessment is a process by which information (could be data or an indicator) is obtained relative to some known objective or goal Kizlik, 2012.The framework built for resilience assessment, whether it is quantitative, qualitative or semi-quantitative, is frequently based on indicators Hosseini et al., 2016;Cantelmi et al., 2021.Quantitative approaches offer domain-agnostic measures to quantify value across applications and structural-based modelling approaches that model domain-specific representations.Semi-quantitative approaches provide a general numerical description of the classification, without detailed formulae or models.Qualitative approaches are related to approaches without a numerical descriptor and based on people's Z. Yang et al. judgments and analysis, like surveyed experts or operators Hosseini et al., 2016;Cantelmi et al., 2021.Furthermore, qualitative approach includes also graphics or illustrations, which present the value or change of indicator and data without numerical calculations.
Resilience assessment is frequently based on indicators Cutter, 2016.To produce knowledge, which can be used to make decisions, and set policies, requires information systems to transform data into information Peris-Mora et al., 2005;Bourgeois, 2014).Moreover, according to Eurostat Eurostat, 2014, indicator-based assessment consists of setting the expected evolution for the indicator by reference.Indicator-based resilience assessment could be therefore defined as an approach to knowing resilience value or level by observing indicators with a set reference.In the context of CIs resilience assessment, an indicator provides information to measure or assess the level of properties, functionality, or capabilities.
An indicator is measurable, associated with a criterion, intended to observe its evolution at defined intervals Yang et al., 2022-a. .A criterion is a character or a sign, which is used to distinguish a thing, a concept or to make a judgment of appreciation.An indicator is objective information, whose assessment, like resilience assessment, could be quantitative, semi-quantitative or qualitative.For example, temperature could be an indicator assessed through objective quantitative methods, such as thermometer readings.Meanwhile, indicator assessment can be based on many subjective judgments, which we call qualitative assessment.For example, for an indicator, "the number of satisfied customers served", the "number" is objective information, but the method of getting this information is based on the subjective judgements of customers.Semi-quantitative indicator assessment, for example, quantifies the number of satisfied customers, which is categorised by level as below per cent and above 50 per cent; Temperatures are classified as below degrees, between 20 and 40 degrees and above 40 degrees.
Moreover, the indicator measure needs reliable data Vogel and Canada, 1997;Cutter, 2016;CORDIS-Smart Resilience Indicators for Smart Critical Infrastructures, 2018).Data is a discrete fact, a raw element or the result of an observation, an acquisition, or a measurement, carried out by a natural or artificial instrument.Data are objective and don't have to function to evaluate or assess an object.For collective work to be done based on data, it is necessary to ensure a unit of format and a standard of measurement.The process that transforms numerous data into actionable indicators is called indicator assessment.
In conclusion, indicator-based resilience assessment for CIs refers to an approach including three factors, CI resilience, Usable indicator, Required data and two phases (see Fig. 3): • resilience assessment: a process in which resilience values are obtained by usable indicators • Indicator assessment: a process in which indicator values are obtained by reliable data

Focused elements
Each existing resilience assessment is based on appropriate steps designed by the creators or users themselves.In designing the steps, many elements of the assessment process need to be specified, such as criteria, indicators, assessment methods, data, indicator weights, etc.According to the current state presented above, this review work will focus on eight elements (see Table 2) of the assessment process in each of the selected papers.Since the assessment approach for CI resilience and indicator has been studied in several current research papers Hosseini et al., 2016;Cantelmi et al., 2021;Gasser et al, 2021;Sun et al. 2020), it is simply categorised into quantitative, qualitative and semiquantitative.

Suitable papers and indicator identification
The online bibliographical databases are becoming an increasingly popular tool to screen scientific literature with key words Macedo-Rouet et al., 2012.The first paper search method is therefore based on an online database, Web of Science.Based on the aims of this study, "Critical infrastructure", "Resilience" and "Indicator" are selected as key words.However, the limit of these keywords would miss some articles in the search scope, for example, a paper describing hospital building as

Table 2
Target elements to be identified in each selected paper.

Table 3
Three review articles of reference search.

Hosseini et al. (2016)
A review of definitions and measures of system resilience.

Meerow et al. (2016)
Defining urban resilience: A review.Landscape and urban planning Curt and Tacnet (2018) Resilience of critical infrastructures: Review and analysis of current approaches "hospital facility" Grimaz et al., 2021.This paper would not be selected through keyword search with "infrastructure".On the other hand, if "hospital facility" were also considered as a key word, the research result would include many articles not related to infrastructures.It illustrates how expanding the search to nearby terms can catch many irrelevant articles.This opinion has been mentioned in the study of Yang et al. (2022-a) in terms of the keyword "indicator".Therefore, a second approach, a reference search based on three recently popular review papers in resilience study Hosseini et al., 2016;Meerow et al., 2016;Curt and Tacnet, 2018), is used for the paper search work.Indeed, some existing tools for reference review, like PRISMA 1 or CORTEX, 2 are able to screen automatically by keywords.But again, for this study, keywords cannot be clearly defined, while the extension of the search to nearby terms would lead to select many unsuitable articles.Consequently, a reference search through manual selection is necessary and inevitable.
Paper searches in this study involve several manual steps, including selecting based on the title and abstract.During the operational process, three people perform these steps, but the whole research team decides the selection criteria.Each result has been discussed several times and approved by the research team.The keywords of suitable article selection are based on the identification of indicators because the indicator is the priority target of this article.Papers that do not contain indicators were excluded from our study.

Keyword Search Method
The Keyword Search Method in this study deploys and adjusts the methodology used in the review paper of Yang et al. (2022-a), which includes three main steps: Key words selection, Database screening and Suitability analysis.An electronic search with keywords "Critical infrastructure", "resilience" and "indicator" was applied on Web of Science on 25 January 2022.The search strings "critical infrastructure", "resilience" and "indicator" were set as "Topic" items, with the conjunction "and" and 128 papers were screened.The suitability analysis, based on the paper abstract, was applied to exclude the papers that match the keywords, but not the research interests.For example, Jeong and An (2016) discuss the usefulness and applicability of risk assessment from the point of view of the weight of key performance indicators (KPI), without description of concrete KPIs.Another example is from Wang et al. (2019) to assess energy infrastructure models from the resilience perspective, not the resilience of energy infrastructures.After excluding articles for which resources could not be found, 22 papers, which are considered to contain resilience indicators for CIs, were selected (see Fig. 4).

Reference Search Method
The reference search used in this study is based on three manual steps, 1) Review Articles Selection, 2) Selection through Title and 3) Suitability Analysis.

Review articles selection
The first consideration is that, due to the widespread use of the term resilience, there are already several review articles studying system resilience or CI resilience.These studies may not have the same purpose as this review study, but they make it possible to collect many references about resilience assessment.Three review articles are useful for this study (see table 3).Many articles were documented and discussed in these three review studies, so their references are browsed in detail and those that do not meet study interest are filtered out.Filtering consists of two steps: Selection through Title and Suitability Analysis.

Selection through Title
This step aims to observe whether the title of the reference contains the three keywords abovementioned for the Keyword Search Method and their synonyms or related vocabularies.All terms and vocabularies are divided in three categories (see table 4) corresponding to the three key sectors mentioned in section 3. The criterion is that the title must contain at least one element from each category.For example, if an article is entitled "Travel time of safety road networks", it is considered to correspond to criteria with its elements: "road networks" as category 1; "safety" as category 2; and "travel time" as category 3. We should point out, that the terms in this table are limitless.All types of "critical infrastructure", all capabilities or properties concerning "resilience" and all terms nearby "indicator" are recognised as relevant.Which terms can be considered relevant relies on the subjective judgement of the screener.Because of this, many words cannot be predicted before the reference search.Automatic searches are unreliable and manual searches are necessary.

Suitability analysis
Similar as the suitability analysis in the Keyword Search Method, this step aims to exclude the papers that don't match the research interests.The suitability analysis in the second method is more complex than the keyword search.In the keyword search, three applicable keywords guarantee every screened paper concerns "indicator", "resilience" and "critical infrastructure".The main task of suitable paper selection in the keyword search is checking whether the indicator mentioned is to assess CI resilience.However, for the second method, with a larger scope (see table 3), more information needs to be checked in reading papers' abstracts: 1) Whether a paper assesses the resilience, or relating capabilities and properties.
This task can be completed based on the definitions and examples listed in section 3.1.The goal is to observe whether papers assess CI capabilities or properties relating to resilience.
• 2) Whether the assessed system is critical infrastructure.
As we only have CI sectors (see table 1), not specific CI types, it is recommended to use the criteria in Moteff et al., (2003) to identify whether the target system is CI or not.Moteff et al., (2003) did not specify which infrastructures should be considered critical but argue "… for an infrastructure to be judged critical it must be vital to one or more broad national functions.That set of functions has expanded over time, beginning with national defence and economic security, to include public health and safety, and then national morale".As long as the system assessed in the paper meets this criterion, we can consider the system to be critical. 1https://www.prisma-statement.org/Default.aspx. 2 https://scirev.org/journal/cortex/.
• 3) Whether a paper applies the "indicator" function to other terms or terminologies, and whether they are used to assess CI resilience.
According to the study by Yang et al. (2022-a), some studies use other terms, which fit the definition of "indicator", for assessment, such as indices, parameter and metrics.Like a second task, which terms or vocabularies should be considered as "indicator" could not be specified.This study focuses on the term or terminology fitting the definition of indicator (i.e., "a chosen piece of information, intended to observe its evolution of something").
With the three mentioned pieces of information, we could finally identify the papers concerning indicator-based resilience assessment for CIs.Consequently, 20 papers were selected after the reference search based on the three review articles (see Fig. 5).Furthermore, in order to analyse more papers, this study applies a second time reference search method based on the 20 selected papers in the first reference search.16 papers were selected after the second reference search (see Fig. 5.The details of suitability analysis are presented in Appendix 1 and 2 (see A.1 and A.2): "X" means that the article is suitable and selected for this study; "repeat" signifies the articles found through the second reference research were also found during the first reference research.

Results
All target elements in each paper relevant to this study are identified and listed in Fig. 6 and in three tables: • Table 5. Results summary for the 22 papers selected through keyword search • Table 6.Results summary for the 20 papers selected through first reference search • Table 7. Results summary for the 16 papers selected through second reference search From left to right, three tables illustrate paper numbers, references, capabilities or properties, temporal stages, relevance of interdependence, relevance of implementing action, identified indicators, suitable CI sectors of identified indicators, resilience assessment approach and indicator assessment approach.The same indicators used by several different documents appear in one column.Due to space limitations, the indicators in the original article have been simplified in this paper.This study only discusses the identified and exportable contents.Elements not discussed in papers are marked as Not Available ("NA").For example, in the case of the first article in Table 5 from Kołowrocki and Soszyńska-Budny (2019), which does not clarify the resilience stages involved, this study prefers to note "NA" to avoid misinterpreting the author's intentions.
"A" means that indicators are suitable for all CI sectors.We highlight that suitable CI analysis of each indicator focuses on the indicator itself, not the CIs mentioned in papers.For example, LaLone et al. ( 2017) applies the indicator "human noise" on the power grid system, but this indicator is suitable for all types of CIs and noted as "A" meaning all sectors of CIs.In some cases, the suitable CI sectors of an identified indicator could be in several sectors, for example, "the days of CI disruption" Murdock et al., 2018 fitting all sectors and "number of passengers" Adjetey-Bahun et al., 2014 suitable for Transportation Systems, Seaport/Harbour and Airport.Moreover, this paper considers that the assessment of resilience is based on a whole indicators system that includes numerous indicators.Therefore, when an indicator is part of an indicator system, the identification of adapted CI sectors depends on the indicator system established by authors.

Results analysis
Before discussing the results, it is necessary to highlight that since resilience and CI are terms that don't have standard definitions, this review study is based on the aspects presented in section 3 and the research method designed by our research team.The discussion focusses on the popularity level in the field, rather than on the concrete quantity of each category in Fig. 6, as this study do not cover all of the existing papers, but is based on the 58 articles identified.

Capabilities or properties (I)
According to an analysis of the number of recurrences, among 125 "capabilities or properties" in total, the four most common words are performance (22 times), safety (11 times), vulnerability (8 times), recovery (8 times), accounting for more than 39 percent of all recurrences.The phenomenon that performance recurs most frequently can be explained by the fact that many articles have described resilience in terms of changes in performance over time.However, resilience is a multi-scale and complex concept Wang et al., 2019, which requires assessment relating to various perspectives and all temporal stages.An assessment aiming only at a part of capacities or stages could not assess CI resilience completely.The majority of investigated papers do not meet this requirement.

Temporal stages (II)
Fig. 6 shows that, except for the articles not concerning temporal stages, the majority of indicators aim at the During Event Stage (38 papers) and Post Event Stage (39 papers) of the resilience scenario.Only six articles focus on all stages of resilience scenarios.This proves that the analysis of preparedness and improvement stages of the resilience scenario is not enough.In particular, NEPS is not discussed in many reviewed papers.This study considers this stage highly relevant to the effectiveness of implementation actions.The lack of discussion of NEPS might be related to the insufficient focus on implementation actions.

Interdependence and implementing action (III and IV)
Moreover, positive or negative effects concerning "interdependencies" and "implemented actions" are not discussed enough in selected papers.Section 3 highlights that damage and implemented actions of one CI could cause effects over time.Positive effects refer to the Effectiveness of actions, which could be reflected in various ways, such as the increase of functionality, the change of functional reduction in speed and degree, and the speed of functional recovery.Negative effects signify: 1) the damage caused by implemented actions to target CI and its connected components in human-environment; and 2) the cascading damage to target CI and connected urban components due to interdependencies.Both positive and negative effects are likely to occur over the long term.Resilience indicators should go beyond the boundaries of one shock event or one CI and help create an ongoing resilience    assessment for a CI, as well as for several interdependent urban components.Therefore, both the aspects concerning CIs' interdependence and effects caused by implemented actions should be taken into account for resilience assessment.However, among these selected papers, only 6 articles mention interdependence, while 13 papers relate to the implementing actions concerning cost, resources, effectiveness and safety of actions.

Indicators (V)
The identification of indicators is a complex process.Firstly, only 30 of the 58 articles use the term "indicator" and specifically explain what they are.Some papers apply the function of "indicator" to its synonyms or related vocabularies, like parameter Hromada and Lukas, 2012;Liu et al.,2021), variable Roe and Schulman, 2012;Murino, et al., 2019;Baroud et al., 2014b;Morlok andChang, 2004, metric (Ghosn et al., 2016;Shafieezadeh and Ivey Burden, 2014;Ouyang and Wang, 2015;Ouyang and Duenas-Osorio, 2014;Vugrin et al., 2010;Henry and Ramirez-Marquez, 2012;Pant et al., 2014;Cavalieri et al., 2012;Ayyub, 2013), index Ghosn et al., 2020;Cimellaro et al. 2016;Ayyub, 2014), etc.These vocabularies are closely associated with indicator-based assessment, but have various definitions in different contexts.If they have the function of "indicator" in reviewed papers, they are presented in Table 5, 6 and 7. On the other hand, some mentioned indicators in reviewed papers are not for assessing resilience.For example, two papers consider "resilience" as an indicator for assessing risk-cost tradeoffs in water resources system Li and Lence, 2017 and for assessing recovery capability in Intermodal Freight Transport Chen and Miller-Hooks, 2012.In these cases, we chose the information assessing "indicator resilience" as indicators.
Sometimes the 'indicators' are not presented in an obvious way.For example, Johnsen and Veen (2013) calculate the scores of three issues relating to safety culture, based on a questionnaire-based survey, and the number of scores could be considered as an indicator.Vugrin ED et al. (2010) use a number of metrics to assess performance, whose values in different periods change into the values of "recovery path" and "recovery effort".Then interviewers present these results to experts, who use them to assess the resilience of CIs.Finally, CI resilience is classified as high, medium or low-level.Therefore, what could be considered as a resilience indicator in the whole assessment process of Vugrin ED et al., (2010) is the "results of interviews", which are an assessment of performance, recovery path and recovery effort.Some papers Faturechi and Miller-Hooks, 2014;Reed et al., 2009;Ip and Wang, 2011;Sun et al., 2006;Omer et al., 2015;Li and Lence, 2017;Miller-Hooks et al., 2012;Winkler et al., 2010;Chang, 2003) provide mathematical formulas to calculate resilience or elastic capacity, and this study records the variables needed for the calculation as an indicator.In some other papers Carvalho et al., 2008;Ouyang and Dueñas-Osorio, 2012-b.;Gilbo, 1993, the indicators were obtained from the authors' analysis.
The results also show that the term "indicator" is sometime misused.Adams et al. (2012) apply "redundancies," "recovery activities" and "travel speed" as indicators, but the former two are not objective enough to be an indicator.This problem occurs in many other papers, such as Shirali et al. (2012), Shirali et al. (2013), LaLone et al. (2017), Liu et al. (2021), Shafieezadeh and Ivey Burden, (2014), etc.The subjective terms like availability, resilience, mitigation and recovery are related to ideal expectations of stakeholders, which cannot be an indicator since it must be objective.Such as the "availability of cranes and berths" indicator in Liu et al. (2021), whether it should correspond to crane volumes or quantity of vehicle berths, since the value that is considered to represent the expected performance of that road infrastructure depends on the subjective judgment of stakeholders.Human judgement for expected performance is based on their means, objectives and results Bescos et al., 1997.Therefore, the non-objective terms are not indicators, but called criteria that are used to distinguish a thing or a concept, and to make a judgment of appreciation.

Suitable CI sectors (VI)
36 papers present the indicators that could be applied to all types of CIs.These indicators are relatively general in comparing the indicators only suitable for specific sectors.Moreover, this study identifies more indicators specific to two CI sectors: transportation systems, Drinking Water and Water Treatment.Specific indicators for 10 sectors are not present in the results: Banking, Commercial Facilities, Critical Manufacturing, Defence, Government Facilities, Industrial Base, Materials and Waste, National Monuments and Icons, and Nuclear Reactors.This result does not indicate a low level of interest in these sectors.For example, although we did not identify indicators applicable only to Industrial Base sectors, the studies on industrial plants could be found in some review papers, like Shirali et al. (2012), andShirali et al. (2013).It can only be said that the indicators they use are also applicable to other CI sectors.A hypothesis is proposed, but it needs to be proven: more specific indicators are needed to assess the infrastructure in several sectors, like transport and water systems, where such needs do not exist in other sectors.Meanwhile, when reading the paper, we found that some CI sectors are indeed less discussed for resilience assessment, especially National Monuments and Icons, Materials and Waste, Banking, and Government Facilities.

Resilience and indicator assessment (VII and VIII)
Most reviewed papers use a quantitative assessment approach, both in the indicator and resilience assessment phases.In the "Resilience assessment" phase, the quantitative approach is 44 times higher than the semi-qualitative method, and 6.3 times higher than the qualitative approach.In the "Indicator assessment" phase, these values were 19.5 and 9.75 respectively.Quantitative assessment is frequently related to modelling, simulation and general mathematic equations, while Qualitative assessment is generally based on questionnaires Vugrin et al., 2010;Shirali et al., 2012;Shirali et al. 2013) and graphical comparisons Pietrucha-Urbanik et al., 2020;Adjetey-Bahun et al., 2014;Carvalho et al., 2008).For example, LaLone et al. ( 2017 The assessment for the same capabilities or properties may have different approaches.One example comes from the reliability assessment of the water supply network, related to three articles.Zhan et al. (2020) and Li and Lence (2007) apply the quantitative assessment method in both resilience and indicator assessment phases.Pietrucha-Urbanik et al. ( 2020) use quantitative methods to calculate the values of two indicators (failure rates and water losses), and then compare graphically the change of indicators to qualitatively assess the reliability of the water supply network during different periods.Besides, for the same capabilities or properties, applied indicators are also different.For example, both relating to CI quality, Reed et al. (2009) present a greater number of indicators, including "inoperability", "wind speed for the hurricane", "capacity", "time in days post-event", etc., while Hromada and Lukas (2012) apply indicators including "delay of breaking resistance", "information and probability of successful communication", "total number of risk", etc.The selection of indicators that are more accurate or practical should be an invaluable issue.
From a psychological point of view, assessing an object is a process that involves judgment and decision-making, which apply certain criteria, principles or standards to form an assessment Sun et al., 2019. For example, Hollnagel (2015) classifies "Potential of Resilience Performance" into five levels: Excellent, Satisfactory, Acceptable, Unacceptable and Deficient.The "Potential of Resilience Performance" is subjective, and can be considered as a criterion that can be used to make judgements about system resilience.However, "criterion" and "criteria" for resilience indicators are discussed only in 12 papers Liu et al., 2021;Upadhyaya et al., 2018;Ghosn et al, 2016;Hashimoto et al., 1982;Kamissoko et al., 2020;Tachaudomdach et al., 2021;Adams et al., 2012;Enjalbert et al., 2011;Curt et al., 2010;Cavalieri et al., 2012;Gilbo, 1993;and Chang, 2003.What's more, in these papers the criteria or indicators may be somewhat confusing, due to the misuse of terms or unclear expressions.For example, Kamissoko et al. (2020) present 6 criteria for three interdependent CIs: "the Quantity of Produced Electricity", "the Number of Incoming Trucks", "the Quantity of Required Electricity", "the Performance", "the Safety" and "the Number of Circulating Trucks".But apart from Performance and Safety, all the other elements could be supposed to be objective information and should be called indicators.Conversely, many expressions are not objective enough to be an indicator, but are used as such.

Necessity of an indicators system
The analysis in this study helps find a significant problem with the existing indicators of CI resilience: existing indicators do not assess resilience according to a uniform and well-defined indicator system.Indicator-based assessment needs an indicators system Shavelson, 1987;Shavelson et al, 1990; UNESCO-LearningPortal).More than just a collection of indicator statistics, indicator systems are usually designed to generate more and more accurate information about conditions Shavelson et al, 1990.Ideally, an indicator system will provide a framework for focus aspects, criteria, and how individual indicator components work together Eurostat, 2014; UNESCO-LearningPortal).However, as can be seen from the results, although these identified indicators are all designed to assess the CI resilience, the indicators systems used by reviewed papers are different due to different indicators, criteria, and assessment methods.In addition, Indicators and criteria are frequently misused.In most of the papers, the resilience assessment is incomplete, as they assess CI's resilience without consideration of various capabilities, and all stages, especially without interdependencies and implementing actions.The problems described above are quite common in indicator-based assessment for CI resilience, which directly leads to results that are not comparable or comprehensive.Comparable assessment results require that the assessment process for different CIs follow uniform criteria and metrics.Comprehensive assessment needs consideration on as many aspects as possible.The establishment of an indicators system aiming at these problems would help improve the indicator-based assessment for CIs resilience.

Limit of aspects
This paper is constructed according to the concepts and aspects presented in part 3.These approaches are presented from a scientific position, founded on a hypothesis that all the aspects presented are compatible with each other.However, these aspects may have a conceptual inconsistency.
To understand and model complex systems, the concept of resilience is often associated with the need for much less centralised control-based, deterministic, and linear approaches.The model grounded on a timeline of a linear transition between stages or levels of system performance or function is one of the examples.However, several mentioned capabilities, such as the capacities in the face of uncertainty and complexity could be contradictory to linear and deterministic models.Being in a complex environment, the performance or function of the infrastructure may be constantly changing at all stages, and is difficult to be presented as an ideal single linear.In addition, for many researchers, function or performance is no longer a priority property for studying the resilience of CIs.Many properties or capabilities that are not based on simple linearity are becoming increasingly popular in complex systems, such as reliability, resource (cost)-effectiveness, and safety culture.
Nevertheless, it is undeniable that deterministic and linear approaches occupy a very important place in resilience studies.Moreover, the opinion, that deterministic and linear approaches could be combined with the capability in the face of uncertainty, is also accepted by many studies.This type of identification allows stakeholders and scientists to quickly select applicable indicators for their situations and find desired assessment methods through the papers reviewed in the study.For example, for an assessment aiming at the PrES of resilience scenario, based on the tables and Fig. 6, 18 relevant articles could easily be found based on the tables 5, 6, and 7, and Fig. 6.For a study focusing on transport resilience indicators, 48 relevant papers could also be quickly found, 36 of which propose indicators applicable to all CI sectors.Moreover, based on the results, future studies could establish an indicator system at a relatively higher level by observing important areas and existing shortcomings.

Limitation of literature review
The limitations of this study concern the literature review and the elements that aren't viable.Firstly, the "reference search method" is applied by noting only the articles in the existing scientific bibliographies.This led to a lack of investigation of the articles published after the three original review papers Table 3.Secondly, the information about capabilities or properties in several papers is missing (noted with NA in Table 5, 6 and 7).For some papers because the article does not address  the information sought, and for others because the message is not clear.
In the spirit of furthering this work, the future idea could be to formalise a criterion and analyse all studies against this, eventually filling the "NA" in Fig. 6.Moreover, the future study could also suggest possible assessment methods for the articles that do not address the information intended.Table 4.

Conclusion
To investigate the existing resilience indicators for CIs, this paper undertook the following steps: -define the elements to be identified for this study by presenting current studies for indicator-based assessment of CI resilience.
-screen the relevant scientific papers with two search methods, including the method for indicators identification; -analyse the target elements based on results.
The results of the study contribute to a comprehensive understanding and application of resilience indicators for CIs.Meanwhile, it provides a clear orientation for further research (i.e., establishing an indicator system for CI resilience assessment).For this goal, several questions are highlighted based on the results of this review work: • since the results show that indicators can be both general and specific, how develop general indicators systems that could be suitable for all CIs, at the same time specific and practical for applying to different CIs and contexts?Should the indicator system be classified into several levels for this issue?• how to develop the general indicator systems that could be suitable for all CIs, which are also specific and practical for applying to different CIs and contexts?Should the indicator system be classified in several levels for this issue?
• how to construct the indicator systems that take into account implementing actions and interdependencies between CIs, internal components and also between CIs and other urban components?• how does the indicator system address the shortcomings of existing indicators, like the lack of assessment relating to the Next Event Preparation Stage; • how to establish efficient criteria for resilience assessment; • how to formalise the methods for "resilience assessment" and "indicator assessment"; All of these questions are worth considering in the future.Funding.
The authors performed this work with funding from the UrbaRiskLab (URL) project (https://urbarisklab.org/fr/ and the RESIIST project (ANR-18-CE39-0018, https://research gi.mines-albi.fr/display/resiist/RESIIST + Home (in French)) that is jointly funded by the French National Research Agency (ANR) and the General Secretary of Defence and National Security (SGDSN).The authors acknowledge these organisations for their support that helped improve the paper.
) draw the change of social indicators based on the data of Tweets per hour that mention keywords and Customers affected by Power Outages during hurricanes.
, this work establishes an initial mapping of resilience indicators and builds a state of the art.For each indicator, this study defines: • the temporal stage of resilience as noted by PrES (Pre-Event Stage), DES (During Event Stage), PoES (Post-Even Stage) and NEPS (New Event Preparation Stage); • the suitable CI sectors of indicators; • the resilience and assessment approaches of indicators.

Table 1
Sectors of CIs.

Table 4
Synonyms or related vocabularies of keywords in three categories.
Z.Yang et al.

Table 5
Results summary for the papers selected through keyword search.

Table 6
Results summary for the paper selected through first reference search.

Table A1
Appendix 1: Papers Selected Through First Reference Search.

Table A2
Appendix 2: Papers Selected Through Second Reference Search.Micro incident analysis framework to assess safety and resilience in the operation of safe critical systems: a case study in a nuclear power plant.