Which data for quantitative landslide susceptibility mapping at operational scale? Case study of the Pays d'Auge plateau hillslopes (Normandy, France)

This paper aims at assessing the impact of the data set quality for landslide susceptibility mapping using multivariate statistical modelling methods at detailed scale. This research is conducted on the Pays d’Auge plateau (Normandy, France) with a scale objective of 1 / 10 000, in order to fit the French guidelines on risk assessment. Five sets of data of increasing quality (considering accuracy, scale fitting, and geomorphological significance) and cost of acquisition are used to map the landslide susceptibility using logistic regression. The best maps obtained with each set of data are compared on the basis of different statistical accuracy indicators (ROC curves and relative error calculation), linear cross correlation and expert opinion. The results highlight that only high-quality sets of data supplied with detailed geomorphological variables (i.e. field inventory and surficial formation maps) can predict a satisfying proportion of landslides in the study area.


Introduction
For natural hazards management, scientists, stakeholders and government authorities need detailed information about the future possible locations of damaging phenomena at a large scale.In France, as in many countries with national risk assessment methods (RAMs) at their disposal, the scale of analysis is imposed by official guidelines.The French RAM (i.e.PPR, Plan de Prevention des Risques) imposes a minimum scale of 1 / 10 000, which was selected to fit the municipality cadastral maps (MATE/METL, 1999).The PPR is divided into three main steps: (1) hazard mapping, (2) vulnerability mapping and (3) risk level mapping.The landslide hazard mapping is performed using a direct geomorphological mapping method based on field observations and available cartographic data.The final hazard maps are the basis for further risk mapping presented in four classes (i.e.very low, low, moderate and high).
The first step of hazard assessment is the susceptibility analysis and mapping (i.e.landslide spatial probability).The susceptibility is the potential location of the landslide source area, combined with the potential runout (Fell et al., 2008).In this study, we will concentrate on the potential landslide source area location that will be named "landslide susceptibility", considering that the study of the landslide propagation will be performed within the hazard mapping step.For the landslide hazard analysis, detailed information on historic records of both landslide occurrences and rainfall or/and earthquake are necessary to determine triggering thresholds.The non-availability of these data often constitutes an operational limitation (Brabb, 1984;Mulder, 1991;Guzzetti et al., 1999;van Westen et al., 2006).Over large areas, these data are difficult to obtain or require important cost and timeconsuming measurements (e.g.field investigations, geophysical measurements, climatic time series analysis, etc.).Therefore, most of the time, the operational hazard maps are susceptibility maps, considered to be "relative hazard maps" (Soriso Valvo, 2002;Guzzetti, 2006).
The direct heuristic (geomorphological) method is the most widely used for establishing official susceptibility and hazard maps in operational contexts (Malet and Maquaire, Published by Copernicus Publications on behalf of the European Geosciences Union.2008).Nevertheless, the indirect susceptibility mapping methods are often considered more objective by scientists.These methods use GIS integrated statistical models based on the spatial relationship between the landslide location and a set of controlling factors, or physically based models analysing the relationships between topographic data and geotechnical parameters, on the basis of infinite slope models (Montgomery and Dietrich, 1994).The indirect methods have been applied and improved by scientists (e.g.Carrara, 1991;Guzzetti et al., 1999;van Westen, 2000;van Westen et al., 2003), but few of these were used in official RAMs (Malet and Maquaire, 2008).However, they can assist in choosing the most revealing data sets to take into account and can be a real alternative in susceptibility/hazard zoning (van Westen et al., 2008;Ercanoglu et al., 2008).These studies are usually conducted at the 1 / 25 000 or 1 / 50 000 scale using the directly available data sets (van Westen et al., 2006;Fell et al., 2008).Even if these spatial statistical approaches (bivariate or multivariate) give good results, a problem of compatibility of the scale of analysis can easily arise between the 1 / 10 000 scale (mapping objective) and the 1 / 25 000 or 1 / 50 000 scale (most accurate data sets available).Some studies have shown the possibility of applying these methods at the 1 / 10 000 scale with an adapted procedure and particular attention to the model calibration (Thiery et al., 2007).Nevertheless, two main steps increase the cost of these approaches and limit their use in operational contexts: (1) the construction of an accurate database and (2) the calibration and validation of the models that are performed using an iterative procedure, since no real standardised methodology exists (Ercanoglu et al., 2004).
These methods are generally developed in complex mountainous environments (for example the Apennines or the Umbria region in Italy, Atkinson and Massari, 1998;Guzzetti et al., 2006).Nevertheless, the plateau and hilly regions of the northwest of Europe are as well affected by slope instability phenomena as, for example, among others, Champagne-Ardennes in France (Mare et al., 2002;van den Eeckhaut et al., 2010), Yorkshire in England (Foster et al., 2007), the Flemish Ardennes in Belgium (van den Eeckhaut., 2006), and the Pays d'Auge in France (Fressard et al., 2010(Fressard et al., , 2011)).Despite the known activity and serious management issues, few scientists have studied them compared to mountain or coastal regions.
This research is conducted on Pays d'Auge plateau hillslopes which are characterised by frequent triggering of shallow landslides.Some attempts to map susceptibility with indirect methods were conducted in this region (Fressard et al., 2010), but remain in the exploratory research framework.However, there is a demand from the stakeholders for obtaining tools that could help in landslide hazard management (CARIP, 2005).It is then necessary to assess the possibility of defining an adapted and operational procedure to map landslide susceptibility using statistical methods in this region.
Instead of comparing different quantification methods of the susceptibility, we propose a comparison of the results obtained with various sets of data with different quality.The variations in the quality of the data sets refer to the resolution and accuracy of the data, but also to the cost (both economical and time spent on the database construction).These two aspects are important for the scale fitting and reliability of the results.
A logistic regression model was used to map the susceptibility as it is considered to be robust by many authors (e.g.Süzen and Doyuran, 2004;Brenning, 2005;van den Eeckhaut et al., 2009;Rossi et al., 2010;Nandi and Shakoor, 2010;Oh et al., 2010;Pradhan and Lee, 2010) and has already given good results in similar hilly environments (van den Eeckhaut et al., 2009;van den Eeckhaut et al., 2010).The method can be directly implemented into GIS software (Kemp et al., 2001;Sawatzky et al., 2009a, b).
The model is run with five sets of data with increasing quality.The improving quality considers all the thematic data, i.e. inventory, landuse data, topographic data and geomorphological data.The statistical performance, but also the general aspects and shapes of the susceptibility classes, are assessed and analysed through the receiver operator characteristic (ROC) curve, the relative error calculation and expert opinion.All final modelled maps are compared on the basis of the previous quality indicators, linear bivariate correlation (V Cramer and Pearson tests) and expert opinion.Finally, the possibility of using the data-driven methods and expected improvements in operational landslide hazard management is discussed.This work does not directly address local stakeholders for immediate application.It rather aims to supply general knowledge between the landslide susceptibility mapping and the quality/resolution of the data sets.

General presentation and geomorphology
The Pays d'Auge is an agricultural region of Normandy of approximately 2500 km 2 (Fig. 1).The main human activity consists in cattle and horse breeding that shaped this typical hedgerow landscape, as in many regions in the northwest of Europe.The Pays d'Auge is a homogeneous geomorphologic entity characterised by a plateau with soft slopes and a massive cuesta constituting its western termination (Fig. 1).The following descriptions are provided for the entire region.
The topography, lithology and hydrology are important environmental factors controlling slope stability (Lautridou, 1971).This region has a maritime temperate climate with a mean annual rainfall around 700 mm regularly distributed over four seasons.Hillslopes are generally not very steep.Only 10 % of the hillslopes have a gradient over 10 • , and 70 % of the area ranked between 5 and 10 • .In the late tertiary and early quaternary, differential erosion shaped the actual topography of the area (Debrand-Passard et al., 1987).
The bedrock is covered with various types of surficial deposits that can be classified into three groups: (1) Very surficial (i.e.around 60 cm) alteration of the marls and clays.(2) Aeolian loess deposits located on the plateau and punctually on the downslope breaks that can reach a thickness of about 10 m. (3) Reworked formations derived from the local substratum alteration (i.e.chalks, clays and sands) and limited displacements due to solifluction processes.This solifluction dynamic is the result of the low mechanical properties of the flint clays, the glauconitic clays and the ferruginous sands, combined with the upslope water discharge.On the upper part of the hillslopes, the surficial formations are often a complex mix of upper flint clays, glauconitic clays and ferruginous sands.This general flow dynamic was initiated during the Quaternary (Lautridou, 1971), and is sometimes still active nowadays (Porcher and Guillopé, 1979).The thickness of the formations is extremely variable and is a function of the upstream materials, water supply and the evolution and age of the process.It is generally between 1 and more than 5 m, but remains difficult to quantify accurately.They are considered to be the most sensitive to landsliding in the study area.
At regional scale, the spatial distribution of the surficial deposits is relatively unknown.As a result, no detailed map was available prior to this study.This point has locally necessitated detailed investigations described in the methodology section of this paper.

Landslide typology and activity
Due to the difficulty of obtaining accurate data and landslide inventories, this study concentrates on a relatively small area (i.e.24 km 2 , Fig. 1).This area is considered to be a test study site that aims to calibrate the methodology and identify the necessary data to expect going further into susceptibility mapping over larger areas.The selection of this study area was guided by the amount of available data, its accessibility and its important landslide activity.The following descriptions of the landslide typology, activity and data collection apply to the 24 km 2 selected study area.
A detailed landslide inventory map was performed during the winters of 2009 and 2010 over the study area (Fressard et al., 2010;Fressard, 2013).Morphology, nature, freshness and size of the scarps and fractures of the landslides were described to estimate the type and relative age of the events.The assessment of the landslide activity consists in four classes as proposed by McCalpin (1984), i.e. active, inactive young, inactive mature and stabilised (Fig. 2).
Two main types of mass movements were identified in the study area, solifluction and landslides (Figs. 2 and 3).No detailed investigations were engaged in the solifluction processes as they were considered inherited from quaternary and dormant or stable processes.For the landslide processes, three main types were observed (Cruden and Varnes, 1996;Maquaire and Malet, 2006): 1.The deep-seated rotational landslides (slups) are large phenomena of an average size of 10 hectares, with slip surface depth estimated between 10 and 30 metres.This type of landslide affects the bedrock by the slipping of upslope chalk panels on the underlying sands and marls.These landslides are dormant or naturally stabilised, as suggested by the smoothed topography and the densely vegetated main scarps (Fig. 3b).
Most of these landslides were mapped under forest.The triggering of these landslides was attributed to different environmental conditions (i.e.Late Tertiary and early Quaternary).Some examples of reactivation can be found in the literature, but often correspond to human actions (excavations, road building, etc.), (Masson, 1976;Brosseau et al., 2011).
2. The shallow translational and rotational landslides are the most frequent types (Fig. 5a).They represent respectively 45 and 20 % of the observed landslides (Fig. 2).They are of a small extent (average size of less than one hectare), with a slip surface of less than 3 metres depth, developed at the contact between the surficial formations and the bedrock.These are single triggered earth slides with a fresh morphology.They are controlled by two main predisposing factors (Fressard, 2013): (1) the slope, with landslides generally occurring at moderate angles (from 5 to 15 %), and (2) the reworked and flowed clayey surficial formations located on mild slopes.Very few landslides of this type were mapped under forest, as the root strength seems to play a stabilising and draining role in surficial formations.
3. The bank shallow landslides are very small landslides of several square metres (Fig. 5).They are observed in the incised parts of channels and are due to the progressive bank undercutting.A small number of these were observed, as the bank erosion and the rapid regaining of the vegetation tends to cover the landslide forms.
These three main types of landslides are triggered and/or reactivated during exceptional episodes of prolonged rainfall in winter, characterised by the rise in the chalk perched  groundwater and by the saturation of the surficial formations.Nevertheless, no detailed investigations on the correlation between event inventories and historical rainfall data or stability modelling were already engaged on the Pays d'Auge plateau.
3 Data sets and methodology

Data sources and data preparation
Different sets of spatial data representing the landslide location and the environmental factors controlling slope stability were acquired on the selected 24 km 2 study area.For the landslide susceptibility analysis, four main types of data are usually employed: (1) landslide inventory, (2) topographic data (e.g.slope angle, slope aspect, slope curvature; extracted from digital elevation models, DEMs), (3) materials data (geology and/or surficial formations) and (4) landuse data.These data can be provided by different institutes and commercial companies or specifically created for the study.The source and the methodology employed in the production of these data sets have a significant impact on the quality/accuracy and on the cost of the data (Glade and Crozier, 2005).For this study, several data set combinations were tested, but only the ones showing a substantial contrast are presented.Five sets of data were finally retained, noted DS in the text.
These five data assemblages were constituted within a constant improvement in the data quality and related cost of acquisition.The data constituting each assemblage are considered homogenous in terms of human and economic investment and can be considered to be used for preliminary (DS 1 and DS 2), intermediate (DS 3) and advanced zoning (DS 4 and 5).The first two sets (DS 1 and DS 2) are composed of free or rapidly produced data.The third set (DS 3) is considered intermediate in terms of cost and accuracy, as only limited field investigations are included (landslide filed mapping).The fourth and fifth sets (DS 4 and DS 5) are composed of data with a high geomorphological significance (surficial formations map) and accurate topographic data (detailed DEM).
A summary of the sources, original scale or grid cell resolution and relative estimated cost for each set of data can be

Landslide data
Three landslide inventories were acquired using different methods.These inventories show large differences regarding the accuracy and number of identified landslides (Figs. 4  and 5).
1.The first landslide inventory was obtained using the directly available BDMvt (French geological survey landslide database, Couëffé et al., 2005).This is a free database that can be directly downloaded from the BRGM website.It was compiled using the landsliderelated incident records, and questionnaires sent to city councils (Couëffé et al., 2005).Only landslides referenced with a high spatial accuracy were used for the analysis.Finally, 61 landslides are identified in the area without any distinction of type and activity.
2. The second landslide inventory was obtained using only the air photo-interpretation (API inventory) on the 2006 and 2009 orthorectified aerial images from the French geographic institute with 50 cm pixel resolution (Fressard et al., 2010;Fressard, 2013).This inventory contains a low number of landslides (i.e. 15 without any distinction of type and activity).The ground vegetation limits the landslide recognition and identification.Moreover, the distinction of type and activity of the landslides is difficult and imprecise on the only basis of the photo interpretation of the ortho-images (insufficient resolution of the images, ground vegetation masking the detailed landforms).3. The third landslide inventory (field inventory) was compiled on the basis of the API inventory, completed by systematic field surveys (Fressard et al., 2010;Fressard, 2013).The landslides were mapped in the field using a cartographic GPS with 1 m accuracy.The landslide boundaries were classified into two zones: (1) a landslide initiation zone and (2) a landslide accumulation zone (Atkinson and Massari, 1998;van den Eeckhaut, 2006;Thiery et al., 2007).Morphological parameters, landslide type and state of activity were stored in a GIS database.This inventory is considered substantially complete (Malamud et al., 2004) and contains 52 mass movement phenomena: 12 solifluction processes, 13 deep-seated landslides and 27 shallow landslides (Figs. 4 and 5, Table 3).Only the shallow landslides were used to prepare the susceptibility map as they are the only type still triggering in the current environmental conditions.The solifluction processes are relatively stable or very slow-moving phenomena.
The deep-seated landslides are dormant phenomena.
Their study would also rather refer to reactivation hazard mapping (Dewitte et al., 2006).

Topographic data
Three DEMs were obtained from different providers and methods.These DEMs allow extraction of the various topographic parameters that would be integrated into the models (e.g.slope angle, slope curvature, slope aspect, etc.).
1.The first DEM is the BDAlti ® , provided for free by the French Geographic Institute (I.G.N.) with a grid cell resolution of 25 m (Figs. 6 and 7).
2. The second DEM was extracted from the digitalised contour lines of the I.G.N. topographic maps at the 1 / 25 000 scale (Figs. 6 and 7) using the modified spline algorithm proposed by the ANUDEM software (Hutchinson, 1996;Hutchinson and Gallant, 2000).Different interpolations were realised and compared.
The best DEM was selected following the procedures of Carrara et al. (1995) and Hutchinson and Gallant (2000).

Geologic materials data
Two maps representing the geologic materials of the study area were acquired (Fig. 6).
1.The 1 / 50 000 scale geological maps (BRGM, French geological survey) were digitalised and classified according to the lithology.2. For the shallow landslide susceptibility mapping, the surficial formations map was considered more relevant than the traditional ground geology map.This surficial formations map was created using extensive field survey and 108 boreholes and augurings of various depths in the study area (81 boreholes available from the BRGM database and 27 specific augurings).These boreholes were interpreted along representative cross sections to identify the rules of deposition and dynamics of the surficial formations.The map was then depicted manually over the study area following the geomorphological interpretation (Fig. 6).Particular attention was paid to the definition of the boundaries of the formations and their link with the topography in order to obtain a map fitting the 1 / 10 000 scale.

Landuse data
Two different landuse maps were tested in this research and show large differences regarding their quality (Fig. 6).
1.The landuse data were obtained throughout the Corine Landcover database provided free of charge by the European Environment Agency (EEA).
2. Because the Corine landcover data is not very accurate (Thiery et al, 2003), interpretation and digitalising on the 2009 orthorectified images of the French Geographic Institute (I.G.N.) were performed to obtain detailed landuse data.Six landuse classes were mapped based on visual interpretation and field checks: forest, grassland, cropland, orchards, fallow fields and urban areas.

Landslide data preparation
The landslide susceptibility modelling necessitates using the landslide data as points feature in the GIS software (Bonham-Carter, 1994;Kemp et al., 2001).BDMvt was directly used to prepare the landslide susceptibility map.The original point format of the database permits direct introduction into the GIS software.For the other landslide inventories (API and field inventory), the original data is  (Poli and Sterlaccini, 2007;Thiery, 2007).This can be performed by selecting only one point for each landslide (generally the geometric centre) or selecting multiple points per landslide.
In this case study, the number of landslide events is relatively low in the API and field inventory (15 to 27 shallow landslides according to the different databases).This is mostly due to the relatively small extent of the study area (24 km 2 ).Modelling landslide susceptibility with such a low number of points trend limits the robustness of the results.Therefore, multiple points per landslide were used to map the susceptibility.The strategy is based on the results obtained by Poli and Sterlaccini (2007).The points representing the landslide locations were seeded within the extent of the landslide depletion zone polygons on a grid representing two times the grid cell size of the raster maps (i.e.20 m).This sampling method is considered to be a reasonable compromise between the necessity of obtaining a sufficient number of points to run a logistic regression model and not oversampling points in the landslide depletion area.All resulting points were used as positives to map the landslide susceptibility.
For the landslide susceptibility mapping, it is important to preserve a set of landslide points to validate the model.For this study, 25 % of the landslides were randomly selected and not used to calibrate the landslide susceptibility model.To avoid effects of oversampling, the selection was performed using the landslide area, not the point data.Then, one landslide cannot be characterised by validation and calibration points.Table 3 summarises the number of landslides and points used in each group for the different landslide inventories.

Selecting the right cell size
A grid cell model was used to map the susceptibility, as it is the most commonly used spatial representation for modelling susceptibility.To compare the models, all thematic layers were resampled at the same cell size and all data perfectly overlap.The choice of the raster images grid cell size was guided by both reference to the imposed cartographic scale and the original scale/resolution of the available data sets.As pointed out by Hengl (2006), no ideal grid resolution exists.This author suggests that the cell size should be the equivalent of 0.0005 × the scale number; i.e. a grid cell should represent a quarter of the maximum location accuracy on the map, usually set at one millimetre (McBratney et al., 2003).In our case, mapping at the 1 / 10 000 scale leads to work with a 5 m cell size.Regarding the original cell size and contour line density on the available thematic maps, it was not possible to use such a detailed grid cell size.The production of the thematic maps will necessitate, for most of them, an important resampling that leads to serious artefacts.The contour lines derived from DEMs are especially sensitive to this issue, as too small a cell size often generates "step" and "sparkling" effects due to the gap between pixel size and the density of the available contour lines (Carrara et al., 1995;Hutchinson and Gallant, 2000).On the basis of several resampling tests, a 10 m cell size (Fig. 7) has been chosen with which to work.This grid cell size is a good compromise between the original cell size of the available raster data and the recommended accuracy for the detailed scale analysis (Florinski and Kurakova, 2000).In this case, one cell will represent the exact maximum location accuracy on the map, which appears fairly accurate.

Modelling method
Logistic regression is the method used in this research.The method is often cited in comparative studies as one of the most efficient data-driven techniques for deriving susceptibility maps (Süzen and Doyuran, 2004;Brenning, 2005;Rossi et al., 2010;Nandi and Shakoor, 2010;Oh et al., 2010;Pradhan and Lee, 2010); moreover, this technique has given good results in similar hilly environments (van den Eeckhaut et al., 2006bEeckhaut et al., , 2009Eeckhaut et al., , 2010)).
Logistic regression describes the relationship between a dichotomous response variable (Y , i.e. the presence or absence of landslides) and a set of explanatory variables (x 1 , x 2 , . . ., x n ).The explanatory variables may be continuous or discrete and do not need a normal frequency distribution.The logistic response function can be written as (Allison, 2001) where p is the spatial probability of occurrence of a landslide, â is the intercept and βi is the coefficient for the independent variable xi estimated by maximum likelihood.More details can be found in Hosmer and Lemeshow (1989), or more specifically for the landslide studies in e.g.Atkinson and Massari (1998) and Ayalew et al. (2005).

Modelling strategy
The statistical model is implemented in ArcGIS 10 ® through the ArcSDM extension (Kemp et al., 2001;Sawatzky et al., 2009aSawatzky et al., , 2009b)).ArcSDM is a "black box" model that does not permit all logistic regression parameters to be set (e.g.sampling of the "no events" stepwise logistic regression), but has the advantage of being GIS integrated.This was preferred to specific statistical software in a transfer of knowledge perspective.The proposed methodology consists in four major steps (Fig. 8).

Selection of the explanatory variables and landslide cell sampling
The explanatory variables of each model were selected according to their accuracy, availability and cost.All collected data were split into the five data sets detailed in Table 2.
In order to preserve a set of landslide data for the validation step, only 75 % of the triggering zone cells were used for the model calibration.The other 25 % were used for the validation step (Chung and Fabbri, 2003).This sampling was performed using a random selection of the landslide polygons.

Model iterations and output maps classification
Successive model iterations were realised with a stepwise introduction of the explanatory variables in order to acquire the best combination based on the statistical performance of the models (Ayalew et al., 2005).As a result, a set of raw probability maps was obtained representing the calculation of each explanatory variable combination in the logistic model.Each map was classified into four susceptibility classes, i.e. very low, low, moderate and high, to match the French RAM official guidelines (MATE/MATL, 1999).
Since all modelled maps were realised using different sets of data (inventory and predisposing factors), it is difficult to define a classification methodology based on equivalent thresholds (i.e.fixed logistic score or proportion of predicted landslides).Then, the classification method was oriented by the equal proportion of the study area.This classification was realised by identifying natural thresholds on the cumulativearea posterior probabilities (CAPP) curve (Bonham-Carter, 1994;Sawatzky et al., 2009aSawatzky et al., , 2009b)).This curve plots the modelled posterior probabilities on a log scale versus the cumulative percentage of the study area (Fig. 9).The raises in the CAPP curve can be used to define class breaks, and the flat sections define the class intervals supported by the data (Bonham-Carter, 1994).Nevertheless, due to the low number of landslide points available to map the susceptibility, defining the same thresholds is not possible, as the cumulative probability curves are often characterised by abrupt changes that do not allow the setting of boundaries of the classification of the expected value (Fig. 9).It is necessary for the visual comparison to obtain susceptibility classes of a relative homogeneous proportion.As it is impossible to obtain the exact same thresholds on the curve, we used closed value ranges to perform the classification (Fig. 9).Then all defined thresholds were set with more or less 5 % variability: null class 65 % (±5 %) of the cumulative ascending proportion of the grid cells, low class 15 % (±5 %), moderate class 12.5 % (±5 %) and high class 7.5 % (±5 %).

Susceptibility maps quality estimation
The accuracy of the models is evaluated using the ROC curves and the area under the ROC curves (AUC).ROC curves plot the "sensitivity" versus the "specificity", where the sensitivity is the proportion of correctly classified known landslide grid cells as unstable, and specificity is the proportion of grid cells outside a mapped landslide that is correctly classified as stable (Metz, 1978;Swets, 1988;Lasko et al., 2005;Beguería, 2006a).The higher the curve is above the diagonal line (corresponding to AUC = 0.5), the better the model is.Then a perfect model fitting would be characterised by an AUC value of 1 and a model not better than random would be characterised by an AUC value of 0.5.In the literature, several empirical classifications of the AUC values were proposed to assess the quality of the model fitting (e.g.Metz, 1978).Traditionally, AUC values under 0.7 reflect a poor performance of the model, values between 0.7 and 0.8 reflect a fair performance of the model, values between 0.8 and 0.9 can be considered to be good, and values above 0.9 can be considered to be excellent.In this study, all landslide-free cells were considered to be negative for the ROC curve computation.
Relative error ξ calculation was performed between the highest susceptibility classes and the response variables to complete the ROC curve analysis.This indicator provides the proportion of landslides mapped outside the high susceptibility class and then gives an indicator of the quality of the classified maps.
where trig is the total number of landslide triggering zone cells, and htrig is the total number of landslide triggering zone cells within the high susceptibility class.
For this study, we globally consider that relative error values above 0.5 cannot be accepted, as a well-defined high susceptibility class should at least be capable of predicting more than half of the observed landslides.
These two quality assessment tests were performed on both calibration and validation landslide data.The results obtained using the calibration data are usually considered overoptimistic in the literature since they were used to prepare the model (Hostmer and Lemeshow, 2000;Chung and Fabri, 2003;Beguería, 2006a).Therefore, the validation data are more suitable for assessing the model performance and more weight should be attributed to the validation data during the analysis and the comparison of the results.

Comparison of the results
The results of each model are then compared on the basis of the obtained accuracy indicators (i.e.ROC curves, relative error), linear cross correlation tests and visual interpretation (expert opinion).Two linear cross correlation tests were computed with the R software (Akgun, 2012).These tests are applied on both unclassified (Pearson's correlation coefficient) and reclassified maps (V Cramer test of association).These tests aim to highlight some general similarities between maps.Both coefficients range from 0 to 1, 0 indicating the absence of correlation and 1 indicating a perfect correlation.
For the expert opinion, we consider that a good susceptibility map should be able to predict a maximum of landslides in the highest class.This class should be as small as possible and should be characterised by a homogeneous zoning.To be accepted, we consider that a susceptibility map should depict a regular and simple zoning.The zones have to be composed of clustered pixels in the same classes and avoid an "isolated pixels effect" generated by the artefacts of the introduced data.

Results
Five final models were successively obtained using the different sets of data (i.e.DS 1, DS 2, DS 3, DS 4 and DS 5).From a general point of view, the results show large variation in the posterior probabilities, regression coefficients, statistical and visual results.These are directly linked to the accuracy, the resolution and the scale fitting of the introduced data sets.The resulting quality indicators (i.e.AUC, relative error and size of the high susceptibility class) are presented in Table 4 and Fig. 10.Table 5 summarises the results of the landslide susceptibility map classification.Tables 6 and 7 show the results of the bivariate statistical tests of association (V Cramer and Pearson coefficients).The graphic result of the modelled maps is presented in Fig. 11, as well as an illustrative zoom showing the final displaying scale (i.e. 1 / 10 000).

Description of the model results
The DS 1 model has an area under the ROC curve for the calibration data set (AUC-cal) of 0.73 and 0.66 for the validation data set (AUC-val).These results indicate a fair to poor classification accuracy (Metz, 1978).The map is very difficult to classify as the susceptibility scores are clustered on a limited number of value ranges.The CAPP curve is then characterised by brutal changes and few thresholds (Fig. 9).Then the relative error is very high: 0.88 for the calibration data (ξ -cal) and 0.92 for the validation data (ξ -val) (Table 4).This means that only 12 and 8 % of the analysed landslides are identified in the high susceptibility class.Some flat (i.e.< 5 %) and valley bottom sectors are classified in the high and moderate susceptibility class, which is not in accordance with the field observations (Fressard, 2013).
For the DS 2 model, AUC-cal and AUC-val are 0.85 and 0.64 respectively (Table 4).This indicates a good to poor classification accuracy.The relative error remains high, with values of 0.44 for ξ -cal and 0.87 for ξ -val.The forest variable, considered to be a stabilising factor (Masson, 1976;Fressard et al., 2011), is not identified by the model.
The introduction of an accurate landslide inventory (DS 3 model) significantly improves the accuracy of the model, which can be considered to be good to fair (AUC-cal = 0.89 and AUC-val = 0.77).The relative error is still high, with a value of 0.47 for ξ -cal and 0.64 for ξ -val (Table 4).In using the field inventory, the zoning is more in accordance with expert opinion and does not indicate high susceptibility levels in flat areas, valley bottoms and forested slopes.Nevertheless, the high susceptibility class is strongly influenced by the slope variable (i.e.classes 10 to 15 % and 15 to 20 %) that leads to a very complex and heterogeneous zoning (Fig. 11).
The use of the detailed surficial formations map in the model DS 4, improves the accuracy of the model.The AUCcal value is 0.92 and the AUC-val value is 0.79 which means that the model has an excellent to fair accuracy.The relative error is decreasing and reaches an acceptable threshold for the ξ -cal i.e. 0.22.The result is better for the ξ -val (0.44) but remains relatively high.In using the surficial formations map the model trends to focus more on the reworked slope deposits and glauconitic sands to determine the high susceptibility scores.This is more in accordance with the expert assumptions (Masson, 1976;Fressard et al., 2010) and this leads to a homogeneous zoning.
The last model, DS 5, is obtained using an accurate DEM.The AUC results are 0.92 for AUC-cal and 0.86 for AUC-val.This means that the model has an excellent to good accuracy.The relative error is 0.30 for ξ -cal and 0.28 for ξ -val, which is the best result obtained.The IFSAR DEM is more adapted to the scale of analysis as it is more accurate and not affected by artefacts due to the contour line interpolation.The obtained susceptibility map takes more into account the local subtle

Landslide susceptibility map comparison
The maps obtained with the DS 1 and DS 2 models are not considered satisfying from both a quantitative and an expert point of view.These results are good for the calibration (i.e. the models identify the introduced landslides well) but lack accuracy for the validation (i.e. the models have difficulty in predicting the location of independent landslides).These results are even more problematic when analysing the classified maps.For calibration and validation relative errors, the maps do not succeed in predicting an acceptable number of landslides in the highest class of susceptibility (Table 4).Visually, the two maps have very complex zoning characterised by serious artefacts and brutal changes in the susceptibility over very small zones (isolated pixel effect).The zoning then appears less realistic and not applicable (Fig. 11).
The DS 3 model is better considering the AUC values.The relative error remains high.Nevertheless, from an expert point of view, the map is more realistic.No flat areas, valley bottoms and forested slopes are identified in the high susceptibility class, which shows the importance of using a complete and field validated inventory.Nevertheless, the zoning remains complex and not easily readable.Both the DS 4 and DS 5 models can be considered satisfying given the quality shown by the output results.Including a detailed surficial formations map leads to a simplified zoning of the high susceptibility class.The accurate mapping of the reworked slope deposits and glauconitic sands increase the predictive power.On both maps, the high susceptibility class is small and predicts a large majority of the observed landslides (Table 4 and Fig. 11).The DS 5 map is of a better overall quality than the DS 4 regarding the statistical accuracy tests.The ξ -cal and AUC-cal results are very close for both maps, but ξ -val and AUC-val are better with respectively 0.28(ξ ) and 0.86 (AUC), which is a clear signal of model improvement.The last model, DS 5, is, from an expert point of view, more realistic due to the better accuracy of the introduced topographic data.
The cross correlation analysis shows a low association between maps on both V Cramer and Pearson's tests (tables 6 and 7).This was confirmed by the visual interpretation and shows that the quality and accuracy of the data are strongly constraining the models (Table 8).The correlation coefficients are much higher in comparing DS 3 and DS 4 (tables 6 and 7).Even if these two maps appear similar with the statistical correlation tests, visual analysis allows identification of obvious differences in the high and moderate susceptibility classes.In this case, the correlation coefficient seems to be influenced by the matching of the very low and low scores on the two maps.

The susceptibility maps and data sets
The landslide inventory is the most important data in the models.The BDMvt of the French geological survey cannot be considered reliable at the 1 / 10 000 scale as it was produced with archive documents and questionnaires to municipalities.Many errors and imprecisions can be identified when compared with detailed field inventories (i.e.several points mapped for the same landslide event, rockfall, collapses or solifluction lobes mapped as landslides, inaccurate location of the landslides, etc.).In this case, the use of API is also inappropriate for the landslide mapping.Few landslides can be identified, and the distinction of type and activity of the landslides is difficult.In this plateau context, only the extensive field inventories can provide satisfying landslide data.The quality of this inventory strongly impacts weights attributed to each predisposing factor and then directly impacts the shape of modelled maps (Ardizzone et al., 2002;Glade and Crozier, 2005;Zêzere et al., 2009).They should then be the most accurate and complete possible.
The models obtained with low and average cost sets of data (DS 1, DS 2 and DS 3) are often affected by artefacts mostly produced by the lack of accuracy of the DEMs.These artefacts are directly imputable to the gap between the available resolution of the thematic layers and the imposed modelling resolution (i.e. 1 / 10 000 scale).The low density of contour lines in this region characterised by a gentle topography forces the integration of smoothing factors during the DEM interpolation.This generally leads to simplified DEM outputs that necessarily propagate in the modelled susceptibility maps.The IFSAR-DEM is not much affected by artefacts compared to the contour line-extracted DEM.This DEM (with an original resolution of 5 m per cell), is more adapted to the 1 / 10 000 scale as it is more able to represent detailed topography, local irregularities and small slope breaks.
The use of the detailed surficial formations map leads to a simplified zoning of the high susceptibility class.Areas within the high class are often the most challenging in the framework of applied mapping and landuse planning discussions.This justifies clearly the interest in using surficial deposit maps to model areas prone to landslides.Furthermore, this data is of a high geomorphological significance, as it adds additional information on the subsurface, which is often lacking in similar landslide susceptibility assessments.
The landuse data is less important for landslide susceptibility mapping.Nevertheless, in this study, we did not assess the potential time variability of the land use.In the Pays d'Auge, the land use has strongly changed during the last 70 years, which constitutes an important limitation in mapping the landslide susceptibility.It should be considered to assess and map the landuse evolution which can be integrated into the susceptibility analysis (Beguería, 2006b;Guns et al., 2012).The introduction of the time variability of the land use necessitates the availability of multi-temporal landslide inventories, as only with these can the landslide occurrence conditions (landuse change) be assessed.Nevertheless, such landslide inventories are difficult to obtain in this case study, as the landslide mapping based only on air photo interpretation cannot be considered effective in the region.
The statistical quality tests have to be used with precaution, especially the values calculated using the calibration data sets, since they are often considered optimistic in assessing the model fitting.More important weight should be put on the validation rate, usually showing worse but more realistic results (Hostmer and Lemeshow, 2000;Chung and Fabbri, 2003;Beguería, 2006a).These tests are interesting indicators of the reliability of the modelled maps, but are not always sufficient (Lobo et al., 2007).To be accepted by local authorities, the maps must be simple and understandable; therefore, the expert validation remains an essential step in the susceptibility analysis.From a regional point of view, this study integrates a larger research project launched in 2009 (Fressard et al., 2010(Fressard et al., , 2011;;Fressard, 2013) into the general description, understanding and hazard mapping of the landslides on the Pays d'Auge plateau.We have demonstrated here the link between the landslide susceptibility and the surficial formations whose mapping can be considered a key point in obtaining reliable hazard maps.

Cost of the data sets
The cost of the data sets is directly correlated with the quality (Table 8).The accurate geomorphological variables have an important cost (Table 1) that strongly impacts the overall cost of the data sets (Table 8).This is especially due to the extensive field campaigns and associated data processing that are necessary for the geomorphological mapping.For the topographic data, the recent progress in remote sensing permits obtaining of accurate data at a reasonable cost.Then, more accurate IFSAR-DEMs are considered cheaper than the usual contour line-extracted DEMs that necessitate long procedures of digitalising and interpolation.Nevertheless, obtaining high-quality maps that can satisfy the enduser demand has an important cost.The cost acquisition of these data is apparently incompressible, as it is mainly due to the necessity for detailed geomorphologic studies.
This study shows the difficulty in obtaining data that are causally related to the landslide predisposing factors.This problem has already been discussed by several authors (Chacon et al., 2006;van Westen et al., 2006;Akgun, 2012).In an operational context, the time, labour and cost associated with the inventory, surficial formations and/or geomorphological mapping often limit the use of specifically created maps for the benefit of directly available data.From an engineering perspective, the pressure to solve a problem in the shortest period of time with the most reliable data in the most economical way forces the use of direct heuristic mapping, which is often considered to be less effective by scientists.On the other hand, the government authorities and end users demand accurate maps that can predict with a high confi-dence the potential occurrence of landslides in their territory.The mapping method, the predisposing factors and the data should then be consciously selected regarding the objectives (scale and expected accuracy) and the amount of money attributed to the study.Nevertheless, for detailed studies claiming to have an operational and statutory purpose, the use of high-quality data is unavoidable.This necessarily leads to an effective cost (Table 8).

The modelling approach: limitations and perspectives
All the proposed modelled maps in this study were realised with a limited number of landslides, which can be considered to be a critical issue for the statistical reliability of the results.This low number of inventoried cases is principally due to the limited extension of the study area given the scale objective of 1 / 10 000.The necessity of supplying the models with high-quality data sets, usually not directly available (especially detailed landslide inventory and surficial formation maps), limits the possibility of working over large areas.
In this stance, we tried to increase artificially the number of training points by using multiple points per landslide.This method was considered efficient in comparative studies (Poli and Sterlaccini, 2007;Thiery, 2007), but was also criticised as it might be a source of spatial autocorrelation (van den Eeckhaut et al., 2006;Atkinson and Massari, 2011).Nevertheless, this appears to be the easiest way of solving the lack of points problem for detailed scale analysis.Moreover, at such a detailed scale, the landslide source areas can be characterised by various types of environmental conditions that cannot be represented using a single point per landslide.It remains then difficult to suggest one or the other method for sampling the landslide data, since few comparative studies are available, and the results might be variable according to the study area-specific conditions.This attempt of applied research into landslide susceptibility was deliberately conducted using GIS integrated statistical software (ARCSDM, Kemp et al., 2001), which does not allow us to take into account all fundamental aspects of statistical modelling, due to its "black box" functioning on logistic regression.For example, it is not possible to set the ratio between "events" and "no events".Recent research shows the importance of the "no event" sampling in the modelling process (Heckmann et al., 2013), which was not taken into account in this study.In this context the recent developments of coupling GIS and statistical software in a "user friendly" process (Brenning, 2007(Brenning, , 2008) ) appears as a key factor to transfer the research advances in landslide susceptibility from scientists to stakeholders.
Similarly, some multi-scale landslide susceptibility or hazard mapping studies have shown the importance of adapting the data to the scale of analysis (Guzzetti et al., 1999;Abella, 2008;Cascini, 2008).This appears quite obvious and well understood from a conceptual point of view.Nevertheless, studies comparing the different sets of data using the same method and the same scale are rare in the literature.In this case, we demonstrate here that using different sets of data, the variability of the quality of the results can be as important as in using different modelling methods.

Conclusions
This study has demonstrated the possibility of assessing landslide susceptibility at the 1 / 10 000 scale with the use of logistic regression in a plateau region of Normandy.This statistical multivariate data-driven method is appropriate for identifying areas prone to landsliding, but necessitates particular attention in the introduced data sets to produce reliable maps at detailed scale.The role played by the predisposing factors must be clearly identified and understood.Without this, the statistical analysis cannot be confidently pursued and produces misleading results (Cascini, 2008;Fell et al., 2008).This research has shown the importance of different key parameters mostly supplied by detailed geomorphologic investigations.These key parameters can be ranked by priority order: (1) the quality of the landslide (event) inventory, (2) the availability of surficial deposit maps, (3) the quality/accuracy of the DEM and (4) the quality of the landuse (change) map.
In using data-driven techniques, the inventory remains the most important parameter because it supplies the model with detailed and reliable observations.
We show that the use of the surficial formations map was very important in identifying areas prone to landsliding and in simplifying the final susceptibility maps.The maps modelled with the surficial formations data are then more homogeneous regarding the zoning and are more in accordance with expert opinion.These susceptibility maps have then more chance of being accepted by end users, as they are more understandable (Thiery et al., 2007).
The topographic data supplied by the DEM is also an important factor to take into account.The direct available DEM in France are still too coarse (i.e. 25 m resolution) to be adapted to the 1 / 10 000 scale.Extracting DEMs from contour maps is a very current technique for obtaining DEM, but necessitates time-consuming procedures of digitalising and interpolation.This is a time-consuming procedure that gives moderate results.We would rather suggest the use of radar DEMs, as they are relatively low-cost images (i.e.±9 Euros km 2 over Europe) and of good quality to work at operational scale.Lidar images are still too expensive to expect specific acquisitions in such an operational context.
For the landuse data, Corine land cover obviously does not fit the expected 1 / 10 000 scale, as they are provided at the 1 / 100 000 scale.Manual photo-interpretation can be a good alternative over small areas.
In any case, the maps obtained with low and moderate cost data cannot be used for statutory mapping framework.These maps have to be carefully presented to the end users as informative maps (Cascini, 2008;Fell et al., 2008).For the Pays d'Auge plateau, it is essential, despite the selected mapping method, to use both field inventory and surficial deposits to obtain suitable susceptibility maps that could be integrated for statutory mapping procedures.
Following this research, discussions were engaged with the BRGM to assess the possibility of producing a regional detailed surficial deposits map.This map should be a powerful tool not only to assess landslide susceptibility over large areas, but also to deal with other types of natural hazards as, for example, swelling and shrinkage of clays.

570M.
Fressard et al.: Which data for quantitative landslide susceptibility mapping at operational scale?

Fig. 2 .
Fig. 2. Observed mass movement characteristics and states of activity of the detailed geomorphological landslide inventory.

Fig. 4 .
Fig. 4. Landslide inventories of the study area.(Note: all landslides mapped by air photo interpretation were confirmed during the field investigations.)

Fig. 5 .
Fig. 5. Example of the different landslide inventories obtained from various sources of investigation.(Note: all landslides mapped by air photo interpretation were confirmed during the field investigations.)

Fig. 7 .
Fig. 7. Example of the three different slope maps generated from the available DEMs in their original cell size resolution and after bilinear resampling at 10 m; detailed zoom on a representative area.(A) Raw BDAlti ® 25 m, (B) raw contour lines DEM 15 m, (C) raw radar DEM 5 m, (D) resampled BDAlti ® slope map 10 m, (E) resampled contour lines DEM slope map 10 m, (F) resampled radar slope map 10 m and (G) location of the illustrative zoom in the study area.(Note: road embankments are not visible on the two first DEMs.)

Fig. 9 .
Fig. 9. CAPP curves of the five models displayed using normalised coefficients and value breaks of the final map classification.

Fig. 10 .
Fig. 10.Receiver operator characteristics (ROC) curves obtained for the different models, (A) calibration data sets and (B) validation data sets.

Table 1 .
Presentation of the source, resolution or scale and estimated cost and accuracy of the available data sets.+ Very low, ++ low, +++ high, ++++ very high, * low cost of acquisition, but timeconsuming procedure, # air-photo interpretation, ♦ specifically created for the study.

Table 2 .
Combination of the sets of data used for each model.

Table 3 .
Summary of the characteristics of the landslide databases used for the susceptibility mapping.

Fressard et al.: Which data for quantitative landslide susceptibility mapping at operational scale? 577 obtained
in a polygon format.Several strategies to transform the polygon into points are proposed in the literature

Table 4 .
Summary of the quality statistics calculated for each landslide susceptibility map (cal = calibration landslide data, val = validation landslide data).

Table 5 .
Proportion of the study area for each susceptibility class of the obtained susceptibility maps and the respective number of landslides.

Table 6 .
Correlation matrix of the linear correlation coefficients (V Cramer) for the final five susceptibility maps, classified probabilities.

Table 7 .
Correlation matrix of the linear correlation coefficients (Pearson coefficients) for the final five susceptibility maps, raw probabilities.

Table 8 .
Cost/accuracy analysis, quality of the model and comments and suggestions for the operational mapping of the five models.(Thevalue of the data accuracy and estimated cost rows is obtained by the sum of the "+" symbols displayed in Table1according to the different data set combinations of each "DS".Minimum reachable value: 4, maximum reachable value: 16.)