The application of frameworks for measuring social vulnerability and resilience to geophysical hazards within developing countries: A systematic review and narrative synthesis

(cid:1) We conducted a systematic review of measuring social vulnerability in LMICs. (cid:1) Few papers measured changes in vulnerability or resilience over time. (cid:1) There was a lack of systematic efforts at validation. (cid:1) Many applications rely on existing frameworks with little adaptation to contexts. (cid:1) More hypothesis-driven studies are required.


Introduction
Geophysical hazards pose significant threats to health, wellbeing, infrastructure and ecosystems across the globe, although there are marked geographical variations in the impacts that they give rise to. Developing nations bear a disproportionate burden of these harms, which is not a function of differential exposure per se, but rather due to limitations in governance, infrastructure, and scientific and technical capacities, as well as high levels of poverty (Pelling, 2003). Moreover, recent years have brought an emerging consensus that hazardous events and disasters are not merely intermittent shocks that leave transient (albeit often substantial) signatures, but rather are events that pose long-lasting challenges to development and poverty alleviation (UNISDR, 2015). Social vulnerability and community resilience 1 have emerged as core concepts for describing the capacity of social systems to prepare, absorb, and adapt to the risks posed by environmental shocks and natural disasters (Adger et al., 2003, Cutter et al., 2008, Klinenberg, 2015, Andrade and Szlafsztein, 2018, Hagenlocher et al., 2018. The growing prominence of these concepts is partly a response to the perceived limitations of historic approaches to disaster risk management, which focussed on technological fixes such as levees, early warning systems, and earthquake-resistant structures (Aldrich, 2012). In particular, analyses of disasters ranging from the 1995 Chicago heat wave (Klinenberg, 2015) to the Japanese tsunami of 2011 (Aldrich and Sawada, 2015) revealed that the quality of an area's ''social infrastructure" may be as important as the state of its physical infrastructure in shaping impacts and recovery trajectories.
In this paper we focus on two core concepts related to social infrastructure: social vulnerability and resilience to geophysical hazards, and on their quantitative measurement within developing nations. These concepts have been studied for more than thirty years. Timmerman (1981) defined vulnerability as the degree to which a social system is adversely impacted by the occurrence of a hazardous event. The nature and extent of the adverse reaction are conditioned by the system's capacity to absorb and recover from the event. From these origins, the field has been marked by important theoretical debates on, for example, how social vulnerability relates to resilience (Bakkensen et al., 2016, Miller et al., 2010, the underlying causal processes through which social vulnerability and hazard exposure interact to generate harm (Birkmann and Van Ginkel, 2006), and even on the definitions of key concepts (Birkmann et al., 2013, Cutter et al., 2003. Moreover, whilst there is broad agreement on the core dimensions that shape social vulnerability (e.g. population growth, income, family structure, etc.) and resilience, the practice of measurement varies significantly (Rufat et al., 2015). Scholars diverge on: whether social vulnerability and resilience are generic properties of systems or are hazard specific; on what indicators best reflect the core dimensions of these constructs; on how those dimensions should be weighted (if at all); and on the preferred analytical techniques for generating composite indexes.
Another core concern is that the majority of frameworks for characterising social vulnerability and resilience have their origins in research in Western, industrialised, high-income nations, and it is questionable whether they can be readily applied to countries across the ''Global South." This is because the processes generating vulnerability or resilience may differ by culture, geography or context, as may the appropriate procedures for measuring them, implying a need for culturally-tailored approaches to framework development and application. This is a critical question at this moment, given that recent years have seen a rapid growth in applications of the social vulnerability and resilience concepts across Asia and within China in particular (Fang et al., 2016. The purposes of such measurement exercises are various (Singh et al., 2017). For example, measurement exercises can inform prioritisation of capacity building exercises, or can shed light on the underlying mechanisms that put individuals and communities at-risk, can reveal how ''vulnerability traps" emerge, or may simply be descriptive mapping exercises.
In summary, the rapidly growing literature that attempts to measure social vulnerability or resilience to geophysical hazards within developing nations is of critical importance, but has yet to 1 We use the term community resilience to emphasise our concern with social systems, as opposed, say, to the resilience of critical infrastructure, or ecological systems. Throughout the paper, when we say resilience, it should be understood as referring to a social system (e.g. community). be systematically synthesised. To address this deficiency, we conducted a systematic review of applications of social vulnerability and resilience frameworks in Lower and Middle Income Countries (LMICs; although we use developing nations, and the Global South as synonyms) using the PRISMA methodology (Liberati et al., 2009). We focussed on LMICs for the reasons outlined above, in addition to our belief the application of such measurement frameworks to industrialised, wealthy countries is a long-standing, wellestablished field, with little need for a new review. We addressed the following research questions: what were the underlying theoretical concepts and frameworks; were the frameworks intended to be generic or hazard-specific; what spatio-temporal scales, data sources, and analytical methods were adopted; what process (if any) was used for adapting the frameworks to the specific application contexts; and were the applications descriptive or hypothesis-driven?

Research methods
To critically appraise and synthesise the measurement of social vulnerability and resilience in LMICs, we applied the systematic review method (Liberati et al., 2009). This differs from the traditional literature review as it applies standardised and replicable methods for data collection and analysis (Petticrew, 2006). In the data collection stage, it extracts items such as peer-reviewed articles, books, papers, and reports using the same search keywords before screening them with the same criteria. In the analysis stage, each item is analysed with the same pre-defined protocol and then synthesised. The approach has its origins in medical and public health research, although it has found increasing application in fields such as ecology (Plummer et al., 2012) and risk research (Thompson et al., 2017) where concerns about objectivity and rigour are similarly paramount.

Data collection
We applied the PRISMA method for data collection (Liberati et al., 2009). In this method, data collection includes the steps of identification (searching and collecting data), screening, and eligibility checking. We used a combination of keywords to identify quantitative empirical studies that measured social vulnerability or resilience to geophysical hazards within low and middle income countries, using the OECD list of Official Development Assistant (ODA) recipients, 2007 2 . We searched for articles published in English between January 1980 and January 2017 using 15 databases (SCOPUS, Web of Science, ScienceDirect, ProQuest, IBSS, ASSIA, Social Service Abstract, EBSCOhost, OpenGrey SIGLE, Pubmed, EMBASE, PsycINFO, Medline, CINAHL and Google Scholar). We used the keywords ''social vulnerability" or ''resilience" combined with ''natural hazard", ''natural disaster", ''environmental hazard", ''climate change" or ''geophysical hazard" 3 to search for studies undertaken within LMICs.
The search identified 2194 articles with an additional 1000 articles identified from Google Scholar 4 . After removing duplicates, 2152 articles remained (Fig. 1). The retrieved studies were further screened manually by reading their titles and abstracts to exclude opinion or review articles, reports, books, working papers, conference papers, theses, qualitative studies, and studies not conducted within LMICs. In addition, the following exclusion criteria were applied: a) Exclude papers that sought to measure perceived (rather than objective) vulnerability; b) Exclude papers focussed on risk assessment (unless social vulnerability or resilience were measured as subcomponents); c) Exclude papers restricted to measuring the vulnerability or resilience of sub-groups of populations, e.g., farmers, coffee growers, children, livestock breeders, etc.; d) Exclude papers focused on capacities to adapt or respond to generalised climate change (e.g. adaptive capacity), but include papers measuring resilience or social vulnerability to climatic hazards (e.g. typhoons, wildfires, floods, etc.) After the screening stage, 111 articles were selected for fulltext assessment for eligibility (using the same criteria as above).
To check the accuracy of the process, we randomly selected 10 articles (9% of the 111 papers), and a second reviewer assessed the full texts independently. Cohen's Kappa coefficient was used to measure the interrater reliability (Cohen, 1968). The higher Kappa score indicates the higher agreement between two reviewers. Usually, scores range from 0.61 to 0.8 can be seen as substantial agreement (Landis and Koch, 1977). In our study, the internal validation Kappa score was 0.78, indicating a high level of agreement on inclusion and exclusion between the reviewers.

Analysis framework
68 articles remained after full-text assessment. These articles were analysed with a pre-defined data extraction form (Appendix A), focussed on the five key research questions outlined in section 1.

Results
Of our 68 selected articles, we noted that work on social vulnerability of geophysical hazards in developing countries has a primary focus on Asian countries, and those that are found in the upper bracket of lower-middle income countries (called upper middle-income countries by ODA) (Table 1). Only 6 articles fell in ODA's least developed countries bracket. Table 2 summarises the data extracted from each article. The following sections present our key findings in relation to the questions posed in section 1.

Hazard types
24 articles measured social vulnerability or resilience with respect to all-hazards, which implies that the former properties are independent of hazard type (Table 3). 19 articles measured vulnerability or resilience to a single hazard, with flooding being the most commonly studied. 25 articles considered multiple hazards. In some cases these multiple hazards were conceptually related (e.g. 9 of the multi-hazard papers studied climatic hazards), in other cases the authors focussed on the set of hazards that frequently occurred within their study areas.

The conceptualisation of social vulnerability and resilience
Conceptualisation is the first step in any measurement exercise. At present, there is no universally agreed definition of resilience or social vulnerability (Roncancio and Nardocci, 2016). Conceptualisation can be thought of as involving two broad processes: (1) defining a broad idea of the object of study and searching for terms to describe it; (2) identifying the dimensions of the concept and their relationships. Below we present our findings in relation to these processes.

Definitions
The 68 papers drew on three primary concepts: social vulnerability (46), resilience (15), and risk (7) (of which social vulnerability was a component). Amongst the social vulnerability papers, the majority (24) considered vulnerability to extend beyond sensitivity to hazardous events and include recovery and adaptation processes (coded as V1 in Table 2 and Table 4). A smaller number (13) focussed on susceptibility, and did not consider recovery or adaptation (coded as V2). For instance, in the study of , vulnerability is defined as ''the characteristic of a place to be wounded and has little capacity to cope [...] without the consideration of resilience (the ability to recover rapidly from disaster)". 7 social vulnerability papers lacked a clear definition of their central concept. On the other hand, when social vulnerability was conceived of within the broader conceptual framework of risk (7 articles), it was typically more narrowly defined to exclude recovery and adaptation.
The resilience articles (15) defined their primary concept with varying terminology, but broadly sharing the idea that resilience is the ability of a system, community or person to prepare, cope with, recover and adapt to a hazard or hazardous event. However, these articles had varying views of how resilience related to social vulnerability. 8 of these papers conceived of vulnerability as the inverse of resilience (coded as R1); 3 papers conceived of vulnerability as a subset of resilience (with the former not considering recovery / adaptation processes) (coded as R2); 2 papers conceived of vulnerability and resilience as overlapping concepts but with independent features (coded as R3); whilst 2 papers did not explicitly define vulnerability, leaving the relationship between the two concepts unclear (coded as R4).     No Yes Table 3 Hazard types studied by the selected papers.  Ahsan and Warner (2014) (2015), Zhou et al. (2015) An interesting pattern is that the social vulnerability papers tended to (implicitly) assume that it was a property independent of hazard types, whilst the resilience articles tended to focus on single hazards or hazard classes (Table 5).

Conceptual framework
The most frequently adopted conceptual framework was Cutter et al.'s (2003) Social Vulnerability Index (SoVI) (referred to by 17 out of 68 studies). Cutter et al.'s (2008) Disaster Resilience of Place model (DROP) was the second most prominently referenced conceptual framework (5 papers). Aside from these two frameworks, a somewhat miscellaneous group of frameworks or theories are referenced as being the authors' point of departure in constructing their frameworks. However, several papers (26) did not explicitly elaborate their conceptual framework, in terms of clearly defining their overarching concept, setting out the underlying dimensions (of social vulnerability or resilience), and explaining how those dimensions (interact to) generate vulnerability or resilience (e.g. some articles simply refer to Cutter et al.'s SoVI model as their inspiration, without explaining its key assumptions and components). We consider these articles to lack a clear conceptual framework.

Geographic unit, data source, and temporal assessment
14 of our selected papers analysed social vulnerability or resilience at the district scale, which is typically the smallest unit in publicly available census data. 23 studies had higher spatial resolutions, such as neighbourhoods, 1 km 2 pixel grids, or households ( Table 6). The remaining 31 papers adopted relatively coarse spatial resolutions, ranging from the town to the nation (including catchments). Table 6 sets out the relationship between unit of analysis and the sources of data in our selected papers. Only 8 articles measured social vulnerability or resilience over multiple years; all of these articles relied upon census data.
Note: Articles in red with underline marked the studies conducted measurement for multiple years.

The methods to assess social vulnerability and resilience
We generalised three approaches to operationalise the conceptual frameworks (social vulnerability or resilience) within our selected papers: theory-driven, data-driven, and mixed approaches (Table 7). Theory-driven approaches use their conceptual framework to guide the selection of indicators (which may necessitate generating novel data e.g. by survey), whilst data-driven approaches generate a list of indicators from available datasets (e.g. census) and screen them for relevance before merging them into factors or components. Among the 42 articles with a clear conceptual framework, the majority (25) applied a theory-driven approach, whilst 11 adopted a mixed approach, and 6 articles were data-driven (which leads to questions about the role of the conceptual framework in the latter articles). Of the 26 articles without a clear conceptual framework, the largest portion was data-driven (11), with 6 adopting a mixed approach, and only 9 articles being theory-driven. It may seem puzzling that any articles lacking a clear conceptual framework could adopt a theory-driven approach. These articles typically justified their indicator selection with reference to standard practices or conventions.
After selecting indicators, the majority of the articles (64) created composite indexes of vulnerability or resilience (''scoring"), using methods such as sum, average, multiply, factor analysis or data envelop analysis (Table 8). In addition, 5 articles conducted a cluster analysis based on scoring. Besides the papers listed in Table 8, 5 articles applied regression without applying any scoring techniques, 1 applied classification, and 2 other articles did not create composite indexes of social vulnerability or resilience.

Validation
Only 8 papers claimed to have validated their frameworks or results. Three general validation approaches were applied. The first validation method compares the results from different approaches to generating composite indexes. For example, Zebardast (2013) conducted a correlation test between the results from their measurement exercise with the results generated by Factor Analysis and Analytical Network Process. The second approach compares estimates of social vulnerability or resilience generated by the authors' frameworks with estimates of those concepts from independent sources. For instance, Boruff and Cutter (2007) sought to validate their measurement of social vulnerability by comparing their results with expert judgments of vulnerability. Using objective measures that are thought to correlate with levels of resilience or social vulnerability following a hazardous event -such as mortality, morbidity, and rebuilding timescales -is the third approach to validation. Four articles in our selected papers used this approach, 3 of which relied upon some measure of economic loss as the validation variable , whilst one used storm damages .

Adaptation to local context
The majority of resilience and social vulnerability and resilience frameworks applied within our selected papers were originally developed in wealthy, industrialised, Western nations. 37 studies (of 68) made an explicit attempt at adaptation to the particular geographic, social or cultural contexts of their study regions. We classify three forms of adaptation: adapting the standard measurement frameworks (i.e. identifying new dimensions, or removing inappropriate ones); adapting the variables selected to represent dimensions within those frameworks; and weighting the dimensions of those frameworks. We discuss these in turn.
Efforts to adapt standard conceptual frameworks were typically modest amendments to SoVI or DROP, or simply ways of implementing those frameworks that were sensitive to particular local contexts. However, the methodology for doing so was typically not made explicit. For example, Roncancio and Nardocci (2016) modified SoVI by adding nine variables relating to sanitation facilities, on the grounds that these are particularly important drivers of social vulnerability in Sao Paulo. Similarly, Gawith et al. (2016) implemented a pared down version of the DROP framework, focussing on variables representing social cohesion, cooperation, social organisation, dynamism and institutional support, on the logic that these constructs are critical drivers of vulnerability in the Fijian context. 6 papers drew on interviews or focus groups involving local communities or elites to screen and select indicators and data sources. For example, working in the Thai context, Siebeneck et al. (2015) drew upon qualitative interviews to identify bespoke indicators for institutional capacity, including the number of Rotary Clubs and the number of monks and novices. Assigning different weights to indicators is another form of contextual adaptation, which 20 articles adopted. Of these, 15 articles consulted with local experts and residents to decide contextdependent weights, whilst 5 studies appealed rather informally to the authors' experience or previous studies as the grounds for assigning weights.

Hypothesis testing
Although the focus of our paper is on the application of frameworks for measuring social vulnerability and resilience, we also considered what research questions, if any, the measurement frameworks were applied to answer. Only 6 of our selected arti-   , ( Hou et al., 2016), Rubin (2014), Siebeneck et al. (2015), Yan and Li (2016), Yang et al. (2015), Yenneti et al. (2016), Zhou et al. (2015), Zou cles tested one or more hypothesis (Akter and Mallick, 2013;Gawith et al., 2016;Zou, 2012;Zhou et al., 2014a;Bene et al., 2016;Rubin, 2014). They posed questions such as under what conditions, and for which hazard types, does social capital contribute to resilience (Bene et al., 2016, Gawith et al., 2016? And are relief funds allocated in ways that prioritise the most vulnerable communities ? The remainder were largely descriptive mapping exercises.

Social vulnerability, resilience and hazard types
Should vulnerability and resilience be conceived of as inherent properties of social systems, or are they better viewed as a function of specific hazards or hazard classes? There is no clear consensus on this within our selected articles. 24 papers measured social vulnerability or resilience in relation to all-hazards; 19 in relation to a single hazard; whilst 25 papers explored multiple hazards. Is the all-hazards approach defensible? Is, for example, resilience plausibly thought of as a generic stock of resources that can be activated and mobilised in the same fashion regardless of the specific hazard in question? After all, droughts, heat waves, earthquakes and so forth differ in terms of their foreseeability, available risk mitigation options, and characteristic damages. For example, agrarian communities facing persistent climatic hazards (e.g. prolonged droughts) will likely have evolved a series of practices, technologies and institutions that are tailored to those specific threatse.g. networks for sharing knowledge and technologies on drought agricultural practices, land tenure institutions, co-operative associations to manage resource exploitation and buffer oscillations in commodity prices, and pooled informal insurance networks (MacGillivray, 2018). The converse is likely to be true for communities exposed to hazards with long historical return periods such as volcanic eruptions volcanic eruptions, earthquakes or tsunamis (England and Jackson, 2011) or where new hazards are triggered due to an unanticipated multi-hazard event. For example, in response to the 2008 Wenchuan earthquake, new debris flow hazards exposed recovering communities to a new suite of previously unknown hazards (Huang and Fan, 2013). In short, whether a society will be resilient (or vulnerable) to a given hazard or set of hazards depends to a large degree on the nature of those hazards, specifically whether there is some inherent adaptive capacity to a particular hazard (see also (Di Baldassarre et al., 2015). The majority of our selected articles measured social vulnerability or resilience in the context of a particular hazard or suite of multihazards. However, there was limited discussion within those articles as to how the specific features of the hazards under consideration had shaped the development of their frameworks.

Concepts, frameworks and adaptation to context
Although there has been a dramatic increase in resiliencerelated research in recent years, the majority of our selected papers relied on social vulnerability as their primary concept (46), rather than resilience (15). Intriguingly, whilst the concept of resilience is often critiqued for its ambiguity, with some arguing that the resultant lack of clarity renders the concept vacuous, there was broad consistency in how it was understood and defined within our papers. Social vulnerability, in contrast, was defined in two different ways in our papers, the distinction turning on whether it was construed as extending beyond the susceptibility to hazards to encompass recovery and adaptation capacities (see section 3.2.2). This observation could be dismissed as ''mere semantics," but we take the view that semantics matter, on the grounds that coherent theory-building and the accumulation of empirical findings requires some basic agreements on core concepts.
Two conceptual frameworks were widely drawn upon, both developed by Cutter and colleagues: the Disaster Resilience of Place Model (DROP), and the Social Vulnerability Index (SoVI). In SoVI, social vulnerability is seen as being comprised of a set of antecedent conditions that shape hazard susceptibility, although Cutter herself later came to view it as inadequately considering structural drivers (Cutter et al., 2008). This motivated the later development of DROP -which extends beyond susceptibility to consider adaptation and coping capacities. The two approaches also differ in that DROP is more of an overarching conceptual framework, whilst SoVI is more operational and can be implemented in relatively routine fashion.
The popularity of SoVI in particular 5 , allied with the fact that it can be readily implemented in cookbook fashion, partly explains why the majority of papers relied upon stock applications of existing frameworks. Where adaptation took place it was typically relatively modest -e.g. tailoring the selection of a specific variable (e.g. proxy  , Sherly et al. (2015) Note: Articles underlined and in red font conducted measurement over multiple years. a These papers also conducted regression analyses. b These papers performed cluster analysis after creating composite indexes. measures for social capital in Thailand) to better reflect particular socio-cultural contexts -and often with little discussion of the underlying method for doing so. Whether this lack of explicit, theorised attempts to tailor frameworks to particular cultural, economic and social contexts is a problem depends on whether one considers that resilience and vulnerability are composed of universal dimensions whose weights are invariant, or views them as operating through more context-dependent processes. Ultimately, this is an empirical question, yet there are strong a priori grounds to suspect that the latter is true, given that there are relatively few, if any, exact laws in the social sciences. Put in less abstract terms, we consider it implausible that the drivers of resilience and vulnerability, their relative importance, and their ideal measures, are likely to be stable across political contexts, cultural setting, scale, and geography (e.g. semi-arid region vs. glacier-dependent river basin). A separate question is the influence of context or geography on measurement error, which is of particular relevance for those papers that relied upon survey data. Put another way, culture may condition how survey questions are interpreted and responded to. For example, De Silva et al. (2006) found that respondents in many developing countries interpreted questions relating to group membership and social support that differed systematically from those of the researchers (e.g. typically interpreting the latter in terms of economic support, rather than emotional or instrumental support). Similarly, Rubin (2015) questions whether the reported high levels of social capital in Vietnam are inflated by systematic measurement error, the idea being that responses to common survey questions intended to measure trust and reciprocity could be biased by a desire on the part of respondents to reflect the official party line (e.g. celebrating needs of society over that of the individual citizen). In the same vein, membership of organisations closely associated with the ruling party in authoritarian regimes may better reflect patronage networks, membership of which is necessary for extracting rents, rather than reflecting trust or respect (Gainsborough, 2007;Rubin, 2015). There was little evidence of considerations of these kinds of sources of measurement error -where social or cultural contexts influence how survey questions are interpreted, or shape incentives for how to respond to them -within our selected papers. Taken as a whole, the foregoing analysis suggests that the development of methodologically sound procedures for tailoring measurement frameworks for specific cultural or social contexts is an important research need.

Gaps in data, methods, and measurements
The reliability, relevance and completeness of data sources used in our selected papers is of particular interest. The majority of papers relied upon census data, with only 15 articles generating their own survey data. Whilst census data typically confers the benefits of standardisation, reliability and broad spatial and temporal coverage, it bears reminding that it is collected for entirely different purposes than the measurement of resilience or vulnerability. As such, there are legitimate questions as to whether the variables commonly collected will be both relevant (i.e. will the variables relate to the theoretical constructs of interest) and complete (in terms of covering all dimensions of vulnerability or resilience). This is particularly true for developing countries, where census exercises are typically resource-constrained, and missing data is often a significant problem. Another issue is that a reliance on census data places a basic constraint on spatial resolution, with district-scale typically being the smallest scale at which data is made publicly available.
Another limitation was the lack of systematic efforts at validation. 8 papers purported to attempt some form of validation, but these were either limited in scope or questionable in approach. For example, expert judgment was used to provide a ''reality check" on estimates of social vulnerability. This approach is in our view simply too informal. For example, what would constitute reasonable agreement between framework estimates and expert judgments of social vulnerability in order for validation to have been achieved? And in instances where experts and framework outputs are viewed as diverging too much, then why should the expert judgments be privileged over the framework outputs, given that the frameworks themselves were presumably developed by scholars laying claim to domain-relevant expertise? Another purported strategy for validation was to determine the extent to which framework outputs were sensitive to alternative (defensible) analytical choices. Whether this can reasonably be understood as validation is questionable -it is more a question of exploring whether analysis results are sensitive to the choice of a particular analytical method. In other words, it is a form of sensitivity analysis. A final approach to validation adopted by some papers was to use objective measures that, by assumption, correlate with levels of resilience or social vulnerability following a hazardous event.
Of the four articles which followed this approach, three relied upon a measure of economic loss as the validation variable. This is illfounded in our view, as it is well established that the most vulnerability populations will have the least amount of assets to lose in a disaster setting, and so we would not expect a strong correlation between vulnerability (or resilience) and economic loss. More reasonable metrics include mortality, morbidity, and rebuilding timescales, although these measures should be seen as providing qualitative corroborations of the frameworks (or identifying discrepancies within them), rather than provide precise quantitative tests.
The clear majority of studies of both resilience and social vulnerability were concerned with mapping out spatial rather than temporal variation in those constructs. This is a significant limitation as resilience, in particular, is broadly conceived of as a dynamic process, which raises questions about the suitability of static measurement approaches (see also Cai et al., 2018). Longitudinal approaches would allow us to learn about the co-evolution of hazards and resilience, and about how the various components of this construct influence each other over time. Another temporal consideration involves forecasting changes in resilience or social vulnerability in future years, for example drawing on demographic projections (see (Hardy and Hauer, 2018). This form of prospective analysis was not undertaken in our selected papers.

Policy-orientation vs. Hypothesis driven studies
A clear majority of our selected papers were concerned with quantifying geographical variations in resilience or social vulnerability. But to what end? The typical claim was that such measurements would be useful for policy or practice, for example in helping prioritise hazard mitigation, emergency response, and recovery efforts. Whether such mapping exercises are actually used by policy makers or practitioners remains an open question. Only 6 of our selected articles were hypothesis driven (section 3.5). What are some open questions that are worth examining in the future? Some relate to environmental justice, i.e. what is the relationship between social vulnerability or resilience and hazard exposure within developing nations, and how is it modified by factors such as hazard type, governance style (e.g. authoritarian vs. clientelistic vs. liberalised), and urban vs. rural? And to what extent is perceived social vulnerability or resilience correlated with 5 Recall that not only was it the most frequently adopted conceptual framework, but it was also referred to as the point of departure by several articles that we classed as lacking a clear conceptual framework (section 3.2.2) ''objective" measures of those constructs across the Global South (e.g. in risk research there has been a long standing concern with the different concepts of risk held by publics and experts, and on the implications of this for policy and governance)? How do resilience and social vulnerability co-evolve in relation to intermittent shocks such as landslide and debris flow events? How effective are various kinds of policy interventions in changing community vulnerability or resilience in developing nations? Does social capital make a particularly significant contribution to resilience in developing countries, given that they typically only have rudimentary social safety nets and limited formal insurance systems?

Conclusion
Before reflecting on future research directions, we first restate our findings: (1) A significant number of studies consider resilience and social vulnerability to be generic properties of social systems, in contrast to a view which sees them as being a function of the specific hazards under consideration; (2) Few papers measured changes in vulnerability or resilience over time, which limits our understanding of the extent to which they co-evolve with exogenous factors (e.g. policy interventions, hazardous events); (3) The majority of studies relied on census data. There are natural trade-offs in relying on census rather than survey data, in terms of spatial scale, reliability, and relevance of the data; (4) Many papers were stock applications of existing frameworks, with little adaptation to socio-cultural contexts. Where adaptation occurred, there was often little discussion of the underlying method for doing so; (5) There was a lack of systematic efforts at validation. Where validation was attempted, it was limited in scope or questionable in approach; (6) More hypothesis-driven studies are required in order to develop a better understanding of the drivers of social vulnerability and resilience, along with the mechanisms through which they shape the capacity to plan, respond, and recover from disasters.
Given the above findings, where should the field go next? Below we identify a series of broad but concrete suggestions for improving the measurement of social vulnerability and resilience in developing countries.
We first suggest that the field should abandon the notion of validation. The reason is simple: SoVI, DROP and cognate frameworks make no quantitative, testable predictions, so cannot be validated under any meaningful sense of the term. Indeed it is no surprise that purported attempts at validation tend to be poorly conceived and implemented. A more promising approach for the field would be to systematically evaluate dimensions of model quality that go beyond those considered in classical validation tests (i.e. that go beyond the traditional emphasis on predictive accuracy or fit to datasets; MacGillivray and Richards, 2015). Key considerations include: are the underlying assumptions made explicit and defended; does the model-building methodology have sufficient ''pedigree;" are parameter and model uncertainties explicitly accounted for (e.g. through sensitivity analysis or ensemble modelling). Many of these quality dimensions are not readily formalised, yet there are structured ways of taking account of them (e.g. van der Sluijs et al., 2005;MacGillivray, 2019;Spiegelhalter and Riesch, 2011).
The second point is that there is a need for a series of reforms if the field is to shift away from a series of largely disconnected, descriptive mapping exercises, towards a field that progressively accumulates empirical findings that support coherent theorybuilding and inform policy-making. This will require the community to develop shared understandings of basic concepts; adopt explicit, defensible methodologies for implementing generic frameworks in particular cultural, social, political and geographic contexts; provide greater emphasis on the formation and testing of hypotheses (see section 4.4 for some concrete suggestions); and adopt more longitudinal studies as opposed to static analyses. This will require sustained, ambitious investments from research funders, and could be catalysed by the development of Resilience or Vulnerability laboratories, where research, expertise, and tools could be shared, perhaps modelled on the NSF Critical Zone Observatory network. Crucially, such efforts, and any such network, should be co-developed along with potential users of this science.
Although much vulnerability and resilience analyses are motivated by a desire to inform practical decision-making, there is limited evidence, at least within our sample papers, that these desires are realised. More rigorous processes of model building and evaluation; more coherent concepts and theory-building; and greater attention to co-production, can help address this mismatch by increasing the policy-relevance and credibility of the science produced within the field.